2024-04-24T23:45:00.009 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T23:45:00.010 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223 --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/orig.config.yaml 2024-04-24T23:45:00.064 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:45:00.162 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672223' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAG+pGj22ncUeVcxlpRyIWqSBqH3a+TcmYezo4zZ2L2EdXf8bPW9wc/huNlOIMsI9HIEwVgZNFDBPEn5Ra2I+0Y= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLnuMpiaSUaozVR5r3sA6dXiKrPWs0j/B5UUR6HuqFjSQFoSF3O5KIAL1c99iZuGWwXMgTwY2aLGgDS8gCwv55g= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+FGEyfBf79U3Jtqg0jVoWnnQHbh8KyUpf31TAH2RE0Bf/hxYEGXIA3BqxdHIejfOQRjw5QG34OdvCJCO6PmWk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T23:45:00.162 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T23:45:00.163 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T23:45:00.163 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T23:45:00.165 INFO:teuthology.task.internal:Checking packages... 2024-04-24T23:45:00.191 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T23:45:00.191 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:45:00.191 INFO:teuthology.packaging:ref: None 2024-04-24T23:45:00.191 INFO:teuthology.packaging:tag: None 2024-04-24T23:45:00.191 INFO:teuthology.packaging:branch: squid 2024-04-24T23:45:00.191 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:45:00.192 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T23:45:00.484 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T23:45:00.485 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T23:45:00.492 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T23:45:00.492 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T23:45:00.544 INFO:teuthology.task.internal:Saving configuration 2024-04-24T23:45:00.564 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T23:45:00.571 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:45:00.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:40:01.196841', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAG+pGj22ncUeVcxlpRyIWqSBqH3a+TcmYezo4zZ2L2EdXf8bPW9wc/huNlOIMsI9HIEwVgZNFDBPEn5Ra2I+0Y='} 2024-04-24T23:45:00.620 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:40:01.194040', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLnuMpiaSUaozVR5r3sA6dXiKrPWs0j/B5UUR6HuqFjSQFoSF3O5KIAL1c99iZuGWwXMgTwY2aLGgDS8gCwv55g='} 2024-04-24T23:45:00.640 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:40:01.195712', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+FGEyfBf79U3Jtqg0jVoWnnQHbh8KyUpf31TAH2RE0Bf/hxYEGXIA3BqxdHIejfOQRjw5QG34OdvCJCO6PmWk='} 2024-04-24T23:45:00.640 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T23:45:00.650 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T23:45:00.650 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T23:45:00.650 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T23:45:00.650 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T23:45:00.765 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f524a350160>, signals=[15]) 2024-04-24T23:45:00.765 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T23:45:00.775 INFO:teuthology.task.internal:Opening connections... 2024-04-24T23:45:00.775 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-04-24T23:45:00.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:45:00.855 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-04-24T23:45:00.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:45:00.931 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-04-24T23:45:00.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:45:01.004 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T23:45:01.012 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-04-24T23:45:01.030 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-04-24T23:45:01.031 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-04-24T23:45:01.087 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-04-24T23:45:01.087 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-04-24T23:45:01.087 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-04-24T23:45:01.087 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:45:01.087 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:45:01.088 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:45:01.089 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-04-24T23:45:01.111 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-04-24T23:45:01.129 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-04-24T23:45:01.129 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:45:01.187 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:45:01.188 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:45:01.188 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:45:01.188 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:45:01.188 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:45:01.188 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-04-24T23:45:01.211 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-04-24T23:45:01.229 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-04-24T23:45:01.230 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-04-24T23:45:01.286 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:45:01.287 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:45:01.288 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-04-24T23:45:01.436 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T23:45:01.448 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T23:45:01.458 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T23:45:01.458 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:45:01.461 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:45:01.463 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:45:01.480 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T23:45:01.490 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T23:45:01.490 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:45:01.518 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:45:01.520 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:45:01.638 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T23:45:01.715 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T23:45:01.736 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-24T23:45:01.737 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-24T23:45:01.737 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:45:01.737 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:45:01.738 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:45:01.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:45:01.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:45:01.755 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-04-24T23:45:01.756 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-04-24T23:45:01.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:45:01.757 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-04-24T23:45:01.812 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:45:01.813 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.14.0-437.el9.x86_64 2024-04-24T23:45:01.813 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2024-04-24T23:45:01.815 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:45:01.815 INFO:teuthology.orchestra.run.smithi173.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:45:01.815 INFO:teuthology.task.kernel:Running kernel on smithi188: 5.14.0-437.el9.x86_64 2024-04-24T23:45:01.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2024-04-24T23:45:01.815 INFO:teuthology.task.kernel:Running kernel on smithi173: 5.14.0-437.el9.x86_64 2024-04-24T23:45:01.816 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum install -y kernel 2024-04-24T23:45:03.898 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-24T23:45:04.059 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T23:45:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-24T23:45:06.130 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-24T23:45:06.477 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-24T23:45:06.724 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-24T23:45:10.602 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-24T23:45:10.647 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-04-24T23:45:10.949 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-24T23:45:12.295 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-04-24T23:45:12.433 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-24T23:45:13.022 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 16 kB 00:01 2024-04-24T23:45:13.599 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-24T23:45:13.862 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-24T23:45:14.968 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-24T23:45:18.663 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-24T23:45:18.943 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-24T23:45:20.115 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-24T23:45:20.494 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:45:20.494 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:45:20.586 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-24T23:45:20.588 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-24T23:45:20.588 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-24T23:45:20.684 DEBUG:teuthology.orchestra.run.smithi110:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:45:20.840 INFO:teuthology.orchestra.run.smithi173.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:45:20.841 INFO:teuthology.orchestra.run.smithi173.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:45:20.932 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-24T23:45:20.933 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-04-24T23:45:20.933 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-24T23:45:21.055 DEBUG:teuthology.orchestra.run.smithi173:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:45:21.184 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:45:18 PM UTC. 2024-04-24T23:45:21.305 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout:Reinstalling: 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 255 k 2024-04-24T23:45:21.318 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 0 2024-04-24T23:45:21.319 INFO:teuthology.orchestra.run.smithi110.stderr:Operation aborted. 2024-04-24T23:45:21.373 INFO:teuthology.orchestra.run.smithi110.stdout:Is this ok [y/N]: 2024-04-24T23:45:21.373 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum reinstall -y kernel || true 2024-04-24T23:45:21.553 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:45:18 PM UTC. 2024-04-24T23:45:21.688 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-24T23:45:21.709 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-24T23:45:21.709 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-24T23:45:21.709 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-24T23:45:21.709 INFO:teuthology.orchestra.run.smithi173.stdout:Reinstalling: 2024-04-24T23:45:21.709 INFO:teuthology.orchestra.run.smithi173.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:45:21.709 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:45:21.709 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-24T23:45:21.709 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-24T23:45:21.710 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:45:21.710 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 255 k 2024-04-24T23:45:21.710 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 0 2024-04-24T23:45:21.710 INFO:teuthology.orchestra.run.smithi173.stderr:Operation aborted. 2024-04-24T23:45:21.768 INFO:teuthology.orchestra.run.smithi173.stdout:Is this ok [y/N]: 2024-04-24T23:45:21.768 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum reinstall -y kernel || true 2024-04-24T23:45:21.864 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:45:18 PM UTC. 2024-04-24T23:45:21.979 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout:Reinstalling: 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 255 k 2024-04-24T23:45:21.980 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 0 2024-04-24T23:45:21.981 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-24T23:45:21.987 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:45:21.987 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:45:22.082 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-24T23:45:22.083 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-04-24T23:45:22.083 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-24T23:45:22.208 DEBUG:teuthology.orchestra.run.smithi188:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:45:22.265 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.6 MB/s | 255 kB 00:00 2024-04-24T23:45:22.265 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:45:22.265 INFO:teuthology.orchestra.run.smithi110.stdout:Total 893 kB/s | 255 kB 00:00 2024-04-24T23:45:22.281 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:04 ago on Wed 24 Apr 2024 11:45:18 PM UTC. 2024-04-24T23:45:22.286 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-24T23:45:22.409 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-24T23:45:22.410 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-24T23:45:22.410 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-04-24T23:45:22.410 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-24T23:45:22.410 INFO:teuthology.orchestra.run.smithi173.stdout:Reinstalling: 2024-04-24T23:45:22.410 INFO:teuthology.orchestra.run.smithi173.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:45:22.410 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:45:22.410 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-24T23:45:22.410 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-04-24T23:45:22.411 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:45:22.411 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 255 k 2024-04-24T23:45:22.411 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 0 2024-04-24T23:45:22.411 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-24T23:45:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-24T23:45:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-24T23:45:22.462 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-24T23:45:22.462 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-24T23:45:22.591 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-24T23:45:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:45:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:22.704 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 11:45:20 PM UTC. 2024-04-24T23:45:22.770 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-24T23:45:22.770 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:45:22.770 INFO:teuthology.orchestra.run.smithi173.stdout:Total 709 kB/s | 255 kB 00:00 2024-04-24T23:45:22.791 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-24T23:45:22.825 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-24T23:45:22.839 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-24T23:45:22.839 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-04-24T23:45:22.839 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-24T23:45:22.839 INFO:teuthology.orchestra.run.smithi188.stdout:Reinstalling: 2024-04-24T23:45:22.839 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:45:22.839 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:45:22.839 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-24T23:45:22.839 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-24T23:45:22.840 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:45:22.840 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 255 k 2024-04-24T23:45:22.840 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 0 2024-04-24T23:45:22.840 INFO:teuthology.orchestra.run.smithi188.stderr:Operation aborted. 2024-04-24T23:45:22.901 INFO:teuthology.orchestra.run.smithi188.stdout:Is this ok [y/N]: 2024-04-24T23:45:22.901 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum reinstall -y kernel || true 2024-04-24T23:45:22.994 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-24T23:45:22.994 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-24T23:45:22.999 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-24T23:45:23.000 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-24T23:45:23.154 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-24T23:45:23.212 INFO:teuthology.orchestra.run.smithi173.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:45:23.212 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:23.339 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:23.339 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:45:23.394 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:45:20 PM UTC. 2024-04-24T23:45:23.509 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-24T23:45:23.510 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-24T23:45:23.510 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:Reinstalling: 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 255 k 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 0 2024-04-24T23:45:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-04-24T23:45:23.869 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:23.869 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:45:24.096 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:24.096 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:45:24.096 INFO:teuthology.orchestra.run.smithi110.stdout:Reinstalled: 2024-04-24T23:45:24.096 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:45:24.096 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:45:24.096 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-24T23:45:24.258 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-437.el9.x86_64.rpm 408 kB/s | 255 kB 00:00 2024-04-24T23:45:24.259 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:45:24.259 INFO:teuthology.orchestra.run.smithi188.stdout:Total 341 kB/s | 255 kB 00:00 2024-04-24T23:45:24.277 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-04-24T23:45:24.302 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:45:24.348 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:45:24.349 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:45:24.349 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:45:24.349 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-24T23:45:24.349 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T23:45:24.377 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-04-24T23:45:24.400 DEBUG:teuthology.parallel:result is None 2024-04-24T23:45:24.466 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-04-24T23:45:24.466 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-04-24T23:45:24.471 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-04-24T23:45:24.472 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-04-24T23:45:24.545 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:24.546 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:45:24.546 INFO:teuthology.orchestra.run.smithi173.stdout:Reinstalled: 2024-04-24T23:45:24.546 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:45:24.546 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:45:24.546 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-24T23:45:24.602 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-04-24T23:45:24.667 INFO:teuthology.orchestra.run.smithi188.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:45:24.668 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:24.706 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:45:24.753 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:45:24.753 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:45:24.753 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:45:24.753 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-24T23:45:24.753 DEBUG:teuthology.orchestra.run.smithi173:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T23:45:24.792 INFO:teuthology.orchestra.run.smithi173.stdout:ttyS1 2024-04-24T23:45:24.813 DEBUG:teuthology.parallel:result is None 2024-04-24T23:45:25.398 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:25.398 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:45:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:45:26.109 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:45:26.109 INFO:teuthology.orchestra.run.smithi188.stdout:Reinstalled: 2024-04-24T23:45:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:45:26.110 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:45:26.110 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-24T23:45:26.283 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:45:26.330 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:45:26.330 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:45:26.330 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:45:26.330 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-24T23:45:26.331 DEBUG:teuthology.orchestra.run.smithi188:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T23:45:26.369 INFO:teuthology.orchestra.run.smithi188.stdout:ttyS1 2024-04-24T23:45:26.389 DEBUG:teuthology.parallel:result is None 2024-04-24T23:45:26.389 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T23:45:26.396 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T23:45:26.396 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:45:26.398 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:45:26.401 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:45:26.444 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T23:45:26.451 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T23:45:26.459 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T23:45:26.459 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:45:26.462 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:45:26.464 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:45:26.576 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T23:45:26.587 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T23:45:26.587 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:45:26.590 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:45:26.593 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:45:26.631 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:45:26.636 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:45:26.642 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:45:26.643 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:45:26.645 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:45:26.653 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:45:26.655 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T23:45:26.736 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T23:45:26.736 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:45:26.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:45:26.741 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:45:26.766 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T23:45:26.775 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T23:45:26.776 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:45:26.807 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:45:26.809 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:45:26.824 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:45:26.906 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:45:26.952 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:45:27.036 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:45:27.076 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:45:27.077 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:45:27.142 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:45:27.185 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:45:27.240 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:45:27.324 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:45:27.368 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:45:27.368 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:45:27.435 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:45:27.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:45:27.528 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:45:27.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:45:27.652 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:45:27.652 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:45:27.718 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-04-24T23:45:27.721 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-04-24T23:45:27.724 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-04-24T23:45:27.761 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:45:27.768 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:45:27.793 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:45:28.299 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T23:45:28.307 INFO:teuthology.task.internal:Starting timer... 2024-04-24T23:45:28.307 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T23:45:28.318 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T23:45:28.330 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T23:45:28.330 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-04-24T23:45:28.389 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-04-24T23:45:28.391 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2024-04-24T23:45:28.459 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2024-04-24T23:45:28.461 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-04-24T23:45:28.543 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-04-24T23:45:28.544 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T23:45:28.545 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-04-24T23:45:28.572 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-04-24T23:45:28.572 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2024-04-24T23:45:28.593 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2024-04-24T23:45:28.593 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-04-24T23:45:28.616 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-04-24T23:45:28.616 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi110.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-04-24T23:45:28.616 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:45:28.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:45:28.642 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:45:28.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:45:28.669 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:45:28.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:45:28.693 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T23:45:28.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:45:28.716 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:45:28.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:45:28.769 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T23:45:28.778 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T23:45:28.796 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T23:45:28.797 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi110.front.sepia.ceph.com,smithi173.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T23:48:58.379 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-04-24T23:48:58.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-04-24T23:48:58.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:48:58.462 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-04-24T23:48:58.502 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-04-24T23:48:58.503 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-24T23:48:58.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:48:58.582 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-04-24T23:48:58.661 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-24T23:48:58.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-24T23:48:58.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:48:58.737 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-04-24T23:48:58.776 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-24T23:48:58.777 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T23:48:58.787 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T23:48:58.787 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:48:58.787 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:48:58.790 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:48:58.790 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:48:58.793 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:48:58.794 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:48:58.819 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:48:58.820 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:48:58.832 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:48:58.833 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:48:58.847 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:48:58.855 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2024-04-24T23:48:58.857 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-04-24T23:48:58.860 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:48:58.864 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2024-04-24T23:48:58.868 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-04-24T23:48:58.877 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:48:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:48:58.884 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-04-24T23:48:58.890 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:48:58.894 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-04-24T23:48:58.895 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:48:58.908 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:48:58.921 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:48:58.943 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:48:58.947 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:48:58.947 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-24T23:48:58.947 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.947 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.947 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.947 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.948 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:48:58.951 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:48:58.952 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-04-24T23:48:58.952 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.952 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.952 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.952 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.977 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:48:58.980 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:48:58.980 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-04-24T23:48:58.980 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.980 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.980 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.980 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:48:58.981 INFO:teuthology.run_tasks:Running task install... 2024-04-24T23:48:58.989 DEBUG:teuthology.task.install:project ceph 2024-04-24T23:48:58.990 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T23:48:58.990 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T23:48:58.990 INFO:teuthology.task.install:Using flavor: default 2024-04-24T23:48:59.001 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T23:48:59.001 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T23:48:59.002 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:48:59.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:48:59.004 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:48:59.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:48:59.006 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:48:59.006 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:48:59.248 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:48:59.248 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:48:59.251 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:48:59.251 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:48:59.265 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:48:59.265 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:48:59.356 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:48:59.356 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:48:59.356 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:48:59.363 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:48:59.363 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:48:59.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:48:59.371 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:48:59.371 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:48:59.371 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:48:59.382 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:48:59.382 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:48:59.388 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:48:59.389 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:48:59.398 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:48:59.398 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:48:59.447 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:48:59.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:48:59.463 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:48:59.520 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:48:59.526 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:48:59.537 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:48:59.583 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-04-24T23:48:59.585 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-04-24T23:48:59.589 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2024-04-24T23:48:59.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2024-04-24T23:48:59.600 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-04-24T23:48:59.602 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-04-24T23:48:59.809 INFO:teuthology.orchestra.run.smithi110.stdout:48 files removed 2024-04-24T23:48:59.814 INFO:teuthology.orchestra.run.smithi173.stdout:48 files removed 2024-04-24T23:48:59.829 INFO:teuthology.orchestra.run.smithi188.stdout:48 files removed 2024-04-24T23:48:59.834 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:48:59.841 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:48:59.856 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:49:00.483 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 217 kB/s | 83 kB 00:00 2024-04-24T23:49:00.563 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-04-24T23:49:00.589 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 176 kB/s | 83 kB 00:00 2024-04-24T23:49:00.809 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 41 kB/s | 11 kB 00:00 2024-04-24T23:49:00.973 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-04-24T23:49:01.005 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-24T23:49:01.119 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-04-24T23:49:01.303 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-24T23:49:01.423 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-24T23:49:01.790 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-04-24T23:49:02.381 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-04-24T23:49:02.910 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-04-24T23:49:03.608 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-24T23:49:04.215 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-24T23:49:08.143 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-04-24T23:49:08.366 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-24T23:49:09.752 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-24T23:49:09.867 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-04-24T23:49:11.657 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-24T23:49:12.796 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 21 MB 00:02 2024-04-24T23:49:16.745 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-24T23:49:17.871 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-24T23:49:18.863 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:49:18.863 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:49:18.874 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:49:18.875 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:49:18.876 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:49:18.877 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:49:18.878 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:49:18.880 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:49:18.881 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:49:18.942 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-24T23:49:18.951 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-04-24T23:49:18.951 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-04-24T23:49:18.951 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-04-24T23:49:18.951 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-04-24T23:49:18.951 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:49:18.951 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:49:18.952 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:49:18.952 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:49:18.952 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:49:18.952 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:49:18.952 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:49:18.952 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:49:18.952 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:49:18.952 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:49:18.953 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:49:18.954 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:49:18.955 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:49:18.956 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:49:18.956 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:49:18.956 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:49:18.956 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:49:18.956 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:49:18.956 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:49:18.956 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:49:18.957 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:49:18.958 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:49:18.959 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:49:18.960 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:49:18.961 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:49:18.962 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:49:18.963 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:49:18.964 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout:Install 148 Packages 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 4 Packages 2024-04-24T23:49:18.965 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:49:18.966 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 220 M 2024-04-24T23:49:18.966 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-04-24T23:49:19.871 INFO:teuthology.orchestra.run.smithi188.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-24T23:49:19.989 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:49:19.989 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:49:20.000 INFO:teuthology.orchestra.run.smithi110.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:49:20.000 INFO:teuthology.orchestra.run.smithi110.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:49:20.001 INFO:teuthology.orchestra.run.smithi110.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:49:20.002 INFO:teuthology.orchestra.run.smithi110.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:49:20.004 INFO:teuthology.orchestra.run.smithi110.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:49:20.006 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:49:20.007 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:49:20.067 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-24T23:49:20.076 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-04-24T23:49:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-24T23:49:20.076 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-04-24T23:49:20.076 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-24T23:49:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:49:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:49:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:49:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:49:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:49:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:49:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:49:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:49:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:49:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:49:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:49:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:49:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:49:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout:Installing weak dependencies: 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout:Install 148 Packages 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 4 Packages 2024-04-24T23:49:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:49:20.090 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 220 M 2024-04-24T23:49:20.090 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-24T23:49:20.179 INFO:teuthology.orchestra.run.smithi188.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 2.8 MB/s | 890 kB 00:00 2024-04-24T23:49:20.254 INFO:teuthology.orchestra.run.smithi188.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.9 MB/s | 145 kB 00:00 2024-04-24T23:49:20.568 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-04-24T23:49:20.713 INFO:teuthology.orchestra.run.smithi188.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 5.2 MB/s | 2.4 MB 00:00 2024-04-24T23:49:20.814 INFO:teuthology.orchestra.run.smithi188.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 4.6 MB/s | 5.4 MB 00:01 2024-04-24T23:49:20.840 INFO:teuthology.orchestra.run.smithi110.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-04-24T23:49:21.048 INFO:teuthology.orchestra.run.smithi188.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-04-24T23:49:21.423 INFO:teuthology.orchestra.run.smithi110.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.5 MB/s | 890 kB 00:00 2024-04-24T23:49:21.431 INFO:teuthology.orchestra.run.smithi188.stdout:(7/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 12 MB/s | 21 MB 00:01 2024-04-24T23:49:21.515 INFO:teuthology.orchestra.run.smithi110.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.5 MB/s | 145 kB 00:00 2024-04-24T23:49:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:(8/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 6.2 MB/s | 4.7 MB 00:00 2024-04-24T23:49:21.656 INFO:teuthology.orchestra.run.smithi188.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 303 kB/s | 25 kB 00:00 2024-04-24T23:49:22.506 INFO:teuthology.orchestra.run.smithi110.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-04-24T23:49:22.781 INFO:teuthology.orchestra.run.smithi188.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 9.7 MB/s | 17 MB 00:01 2024-04-24T23:49:22.832 INFO:teuthology.orchestra.run.smithi110.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.6 MB/s | 1.5 MB 00:00 2024-04-24T23:49:22.865 INFO:teuthology.orchestra.run.smithi188.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.6 MB/s | 222 kB 00:00 2024-04-24T23:49:22.982 INFO:teuthology.orchestra.run.smithi188.stdout:(12/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 6.6 MB/s | 10 MB 00:01 2024-04-24T23:49:23.007 INFO:teuthology.orchestra.run.smithi188.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 190 kB/s | 27 kB 00:00 2024-04-24T23:49:23.049 INFO:teuthology.orchestra.run.smithi188.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 475 kB/s | 32 kB 00:00 2024-04-24T23:49:23.099 INFO:teuthology.orchestra.run.smithi188.stdout:(15/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 7.9 MB/s | 745 kB 00:00 2024-04-24T23:49:23.124 INFO:teuthology.orchestra.run.smithi188.stdout:(16/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-24T23:49:23.166 INFO:teuthology.orchestra.run.smithi188.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.9 MB/s | 127 kB 00:00 2024-04-24T23:49:23.216 INFO:teuthology.orchestra.run.smithi188.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.3 MB/s | 501 kB 00:00 2024-04-24T23:49:23.283 INFO:teuthology.orchestra.run.smithi188.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 677 kB/s | 45 kB 00:00 2024-04-24T23:49:23.359 INFO:teuthology.orchestra.run.smithi188.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.7 MB/s | 133 kB 00:00 2024-04-24T23:49:23.436 INFO:teuthology.orchestra.run.smithi188.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-24T23:49:23.457 INFO:teuthology.orchestra.run.smithi110.stdout:(6/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-24T23:49:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 4.2 MB/s | 321 kB 00:00 2024-04-24T23:49:23.603 INFO:teuthology.orchestra.run.smithi188.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.2 MB/s | 299 kB 00:00 2024-04-24T23:49:23.679 INFO:teuthology.orchestra.run.smithi188.stdout:(24/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 10 MB/s | 5.3 MB 00:00 2024-04-24T23:49:23.704 INFO:teuthology.orchestra.run.smithi188.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 994 kB/s | 99 kB 00:00 2024-04-24T23:49:23.737 INFO:teuthology.orchestra.run.smithi188.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-24T23:49:23.740 INFO:teuthology.orchestra.run.smithi110.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 5.2 MB/s | 4.7 MB 00:00 2024-04-24T23:49:23.804 INFO:teuthology.orchestra.run.smithi188.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-24T23:49:23.880 INFO:teuthology.orchestra.run.smithi188.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 423 kB/s | 32 kB 00:00 2024-04-24T23:49:23.964 INFO:teuthology.orchestra.run.smithi188.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.6 MB/s | 141 kB 00:00 2024-04-24T23:49:24.098 INFO:teuthology.orchestra.run.smithi188.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 7.9 MB/s | 3.1 MB 00:00 2024-04-24T23:49:24.348 INFO:teuthology.orchestra.run.smithi188.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 9.2 MB/s | 3.5 MB 00:00 2024-04-24T23:49:24.440 INFO:teuthology.orchestra.run.smithi188.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.7 MB/s | 249 kB 00:00 2024-04-24T23:49:24.515 INFO:teuthology.orchestra.run.smithi188.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 658 kB/s | 49 kB 00:00 2024-04-24T23:49:24.582 INFO:teuthology.orchestra.run.smithi188.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 230 kB/s | 15 kB 00:00 2024-04-24T23:49:24.674 INFO:teuthology.orchestra.run.smithi188.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.8 MB/s | 264 kB 00:00 2024-04-24T23:49:24.791 INFO:teuthology.orchestra.run.smithi188.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-24T23:49:24.810 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-24T23:49:24.866 INFO:teuthology.orchestra.run.smithi188.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 243 kB/s | 18 kB 00:00 2024-04-24T23:49:25.075 INFO:teuthology.orchestra.run.smithi188.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-04-24T23:49:25.292 INFO:teuthology.orchestra.run.smithi188.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 189 kB/s | 80 kB 00:00 2024-04-24T23:49:25.484 INFO:teuthology.orchestra.run.smithi188.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-24T23:49:25.599 INFO:teuthology.orchestra.run.smithi110.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 5.5 MB/s | 10 MB 00:01 2024-04-24T23:49:25.649 INFO:teuthology.orchestra.run.smithi110.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 505 kB/s | 25 kB 00:00 2024-04-24T23:49:25.667 INFO:teuthology.orchestra.run.smithi188.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-04-24T23:49:25.826 INFO:teuthology.orchestra.run.smithi188.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-24T23:49:25.934 INFO:teuthology.orchestra.run.smithi188.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 985 kB/s | 106 kB 00:00 2024-04-24T23:49:26.026 INFO:teuthology.orchestra.run.smithi188.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-04-24T23:49:26.051 INFO:teuthology.orchestra.run.smithi188.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-24T23:49:26.135 INFO:teuthology.orchestra.run.smithi188.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-24T23:49:26.152 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 16 kB 00:00 2024-04-24T23:49:26.160 INFO:teuthology.orchestra.run.smithi188.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-24T23:49:26.353 INFO:teuthology.orchestra.run.smithi188.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-04-24T23:49:26.386 INFO:teuthology.orchestra.run.smithi188.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-24T23:49:26.412 INFO:teuthology.orchestra.run.smithi188.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-24T23:49:26.437 INFO:teuthology.orchestra.run.smithi188.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-24T23:49:26.470 INFO:teuthology.orchestra.run.smithi188.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-24T23:49:26.496 INFO:teuthology.orchestra.run.smithi188.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-04-24T23:49:26.546 INFO:teuthology.orchestra.run.smithi188.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-24T23:49:26.596 INFO:teuthology.orchestra.run.smithi188.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-24T23:49:26.647 INFO:teuthology.orchestra.run.smithi188.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-24T23:49:26.672 INFO:teuthology.orchestra.run.smithi188.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-24T23:49:26.697 INFO:teuthology.orchestra.run.smithi188.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-24T23:49:26.731 INFO:teuthology.orchestra.run.smithi188.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-24T23:49:26.756 INFO:teuthology.orchestra.run.smithi188.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-04-24T23:49:26.773 INFO:teuthology.orchestra.run.smithi188.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 608 kB/s | 25 kB 00:00 2024-04-24T23:49:26.798 INFO:teuthology.orchestra.run.smithi188.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-24T23:49:26.824 INFO:teuthology.orchestra.run.smithi188.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-24T23:49:26.849 INFO:teuthology.orchestra.run.smithi188.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-24T23:49:26.874 INFO:teuthology.orchestra.run.smithi188.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-24T23:49:27.000 INFO:teuthology.orchestra.run.smithi188.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-24T23:49:27.083 INFO:teuthology.orchestra.run.smithi188.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-24T23:49:27.125 INFO:teuthology.orchestra.run.smithi188.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-24T23:49:27.159 INFO:teuthology.orchestra.run.smithi188.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-24T23:49:27.184 INFO:teuthology.orchestra.run.smithi188.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T23:49:27.251 INFO:teuthology.orchestra.run.smithi188.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-24T23:49:27.301 INFO:teuthology.orchestra.run.smithi188.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-24T23:49:27.376 INFO:teuthology.orchestra.run.smithi188.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-04-24T23:49:27.436 INFO:teuthology.orchestra.run.smithi188.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T23:49:27.487 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-24T23:49:27.519 INFO:teuthology.orchestra.run.smithi188.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-04-24T23:49:27.544 INFO:teuthology.orchestra.run.smithi188.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-24T23:49:27.570 INFO:teuthology.orchestra.run.smithi188.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-04-24T23:49:27.595 INFO:teuthology.orchestra.run.smithi188.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-24T23:49:27.629 INFO:teuthology.orchestra.run.smithi188.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-24T23:49:27.654 INFO:teuthology.orchestra.run.smithi188.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-24T23:49:27.712 INFO:teuthology.orchestra.run.smithi188.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-24T23:49:27.733 INFO:teuthology.orchestra.run.smithi110.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 3.9 MB/s | 17 MB 00:04 2024-04-24T23:49:27.788 INFO:teuthology.orchestra.run.smithi188.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-24T23:49:27.813 INFO:teuthology.orchestra.run.smithi188.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-24T23:49:27.838 INFO:teuthology.orchestra.run.smithi188.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T23:49:27.864 INFO:teuthology.orchestra.run.smithi188.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-24T23:49:27.889 INFO:teuthology.orchestra.run.smithi188.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-24T23:49:27.914 INFO:teuthology.orchestra.run.smithi188.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-24T23:49:27.940 INFO:teuthology.orchestra.run.smithi188.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-24T23:49:27.991 INFO:teuthology.orchestra.run.smithi110.stdout:(11/152): ceph-common-19.0.0-2445.gd2c45d97.el9 2.9 MB/s | 21 MB 00:07 2024-04-24T23:49:28.015 INFO:teuthology.orchestra.run.smithi188.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-04-24T23:49:28.016 INFO:teuthology.orchestra.run.smithi110.stdout:(12/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 784 kB/s | 222 kB 00:00 2024-04-24T23:49:28.040 INFO:teuthology.orchestra.run.smithi188.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 546 kB/s | 54 kB 00:00 2024-04-24T23:49:28.050 INFO:teuthology.orchestra.run.smithi110.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 464 kB/s | 27 kB 00:00 2024-04-24T23:49:28.084 INFO:teuthology.orchestra.run.smithi110.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 473 kB/s | 32 kB 00:00 2024-04-24T23:49:28.090 INFO:teuthology.orchestra.run.smithi188.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-04-24T23:49:28.141 INFO:teuthology.orchestra.run.smithi188.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-04-24T23:49:28.201 INFO:teuthology.orchestra.run.smithi110.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.4 MB/s | 163 kB 00:00 2024-04-24T23:49:28.276 INFO:teuthology.orchestra.run.smithi110.stdout:(16/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.2 MB/s | 745 kB 00:00 2024-04-24T23:49:28.301 INFO:teuthology.orchestra.run.smithi110.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.2 MB/s | 127 kB 00:00 2024-04-24T23:49:28.435 INFO:teuthology.orchestra.run.smithi110.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.1 MB/s | 501 kB 00:00 2024-04-24T23:49:28.502 INFO:teuthology.orchestra.run.smithi110.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 678 kB/s | 45 kB 00:00 2024-04-24T23:49:28.558 INFO:teuthology.orchestra.run.smithi188.stdout:(93/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.0 MB/s | 48 MB 00:06 2024-04-24T23:49:28.569 INFO:teuthology.orchestra.run.smithi110.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.0 MB/s | 133 kB 00:00 2024-04-24T23:49:28.583 INFO:teuthology.orchestra.run.smithi188.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 500 kB/s | 221 kB 00:00 2024-04-24T23:49:28.625 INFO:teuthology.orchestra.run.smithi188.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-24T23:49:28.642 INFO:teuthology.orchestra.run.smithi188.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 430 kB/s | 36 kB 00:00 2024-04-24T23:49:28.644 INFO:teuthology.orchestra.run.smithi110.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-24T23:49:28.676 INFO:teuthology.orchestra.run.smithi188.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-24T23:49:28.701 INFO:teuthology.orchestra.run.smithi188.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-04-24T23:49:28.726 INFO:teuthology.orchestra.run.smithi188.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T23:49:28.743 INFO:teuthology.orchestra.run.smithi188.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-24T23:49:28.744 INFO:teuthology.orchestra.run.smithi110.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.1 MB/s | 321 kB 00:00 2024-04-24T23:49:28.777 INFO:teuthology.orchestra.run.smithi188.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-04-24T23:49:28.853 INFO:teuthology.orchestra.run.smithi110.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.7 MB/s | 299 kB 00:00 2024-04-24T23:49:28.936 INFO:teuthology.orchestra.run.smithi188.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-24T23:49:28.952 INFO:teuthology.orchestra.run.smithi188.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 107 kB/s | 22 kB 00:00 2024-04-24T23:49:28.953 INFO:teuthology.orchestra.run.smithi110.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 994 kB/s | 99 kB 00:00 2024-04-24T23:49:29.021 INFO:teuthology.orchestra.run.smithi110.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-24T23:49:29.203 INFO:teuthology.orchestra.run.smithi188.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-04-24T23:49:29.454 INFO:teuthology.orchestra.run.smithi110.stdout:(26/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 4.6 MB/s | 5.3 MB 00:01 2024-04-24T23:49:29.469 INFO:teuthology.orchestra.run.smithi188.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-24T23:49:29.521 INFO:teuthology.orchestra.run.smithi110.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-24T23:49:29.537 INFO:teuthology.orchestra.run.smithi188.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 529 kB/s | 309 kB 00:00 2024-04-24T23:49:29.579 INFO:teuthology.orchestra.run.smithi110.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 548 kB/s | 32 kB 00:00 2024-04-24T23:49:29.638 INFO:teuthology.orchestra.run.smithi110.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.4 MB/s | 141 kB 00:00 2024-04-24T23:49:29.645 INFO:teuthology.orchestra.run.smithi188.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 773 kB/s | 548 kB 00:00 2024-04-24T23:49:29.688 INFO:teuthology.orchestra.run.smithi110.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.6 MB/s | 3.1 MB 00:00 2024-04-24T23:49:29.795 INFO:teuthology.orchestra.run.smithi188.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-04-24T23:49:29.988 INFO:teuthology.orchestra.run.smithi188.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-24T23:49:30.146 INFO:teuthology.orchestra.run.smithi188.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 192 kB/s | 67 kB 00:00 2024-04-24T23:49:30.330 INFO:teuthology.orchestra.run.smithi188.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-04-24T23:49:30.405 INFO:teuthology.orchestra.run.smithi188.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 362 kB/s | 151 kB 00:00 2024-04-24T23:49:30.472 INFO:teuthology.orchestra.run.smithi110.stdout:(31/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 10 MB/s | 48 MB 00:04 2024-04-24T23:49:30.539 INFO:teuthology.orchestra.run.smithi110.stdout:(32/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.9 MB/s | 3.5 MB 00:00 2024-04-24T23:49:30.565 INFO:teuthology.orchestra.run.smithi110.stdout:(33/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.6 MB/s | 249 kB 00:00 2024-04-24T23:49:30.598 INFO:teuthology.orchestra.run.smithi110.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 838 kB/s | 49 kB 00:00 2024-04-24T23:49:30.624 INFO:teuthology.orchestra.run.smithi110.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 262 kB/s | 15 kB 00:00 2024-04-24T23:49:30.674 INFO:teuthology.orchestra.run.smithi110.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.4 MB/s | 264 kB 00:00 2024-04-24T23:49:30.724 INFO:teuthology.orchestra.run.smithi110.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 366 kB/s | 18 kB 00:00 2024-04-24T23:49:30.747 INFO:teuthology.orchestra.run.smithi188.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-24T23:49:30.814 INFO:teuthology.orchestra.run.smithi188.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-24T23:49:30.832 INFO:teuthology.orchestra.run.smithi110.stdout:(38/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.6 MB/s | 762 kB 00:00 2024-04-24T23:49:30.858 INFO:teuthology.orchestra.run.smithi110.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 601 kB/s | 80 kB 00:00 2024-04-24T23:49:30.939 INFO:teuthology.orchestra.run.smithi188.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 900 kB/s | 548 kB 00:00 2024-04-24T23:49:30.958 INFO:teuthology.orchestra.run.smithi110.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-24T23:49:31.000 INFO:teuthology.orchestra.run.smithi110.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-24T23:49:31.014 INFO:teuthology.orchestra.run.smithi188.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 120 kB/s | 32 kB 00:00 2024-04-24T23:49:31.026 INFO:teuthology.orchestra.run.smithi110.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-24T23:49:31.076 INFO:teuthology.orchestra.run.smithi110.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-24T23:49:31.081 INFO:teuthology.orchestra.run.smithi188.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-24T23:49:31.101 INFO:teuthology.orchestra.run.smithi110.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-24T23:49:31.135 INFO:teuthology.orchestra.run.smithi110.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-24T23:49:31.160 INFO:teuthology.orchestra.run.smithi110.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-24T23:49:31.185 INFO:teuthology.orchestra.run.smithi110.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-24T23:49:31.319 INFO:teuthology.orchestra.run.smithi110.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 4.5 MB/s | 7.4 MB 00:01 2024-04-24T23:49:31.365 INFO:teuthology.orchestra.run.smithi188.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-04-24T23:49:31.511 INFO:teuthology.orchestra.run.smithi110.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 558 kB/s | 107 kB 00:00 2024-04-24T23:49:31.523 INFO:teuthology.orchestra.run.smithi188.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 704 kB/s | 358 kB 00:00 2024-04-24T23:49:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-24T23:49:31.561 INFO:teuthology.orchestra.run.smithi110.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-04-24T23:49:31.587 INFO:teuthology.orchestra.run.smithi110.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-04-24T23:49:31.590 INFO:teuthology.orchestra.run.smithi188.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 596 kB/s | 303 kB 00:00 2024-04-24T23:49:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-04-24T23:49:31.632 INFO:teuthology.orchestra.run.smithi188.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-24T23:49:31.646 INFO:teuthology.orchestra.run.smithi110.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-24T23:49:31.679 INFO:teuthology.orchestra.run.smithi110.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-04-24T23:49:31.713 INFO:teuthology.orchestra.run.smithi110.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-04-24T23:49:31.755 INFO:teuthology.orchestra.run.smithi110.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-24T23:49:31.788 INFO:teuthology.orchestra.run.smithi110.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-04-24T23:49:31.956 INFO:teuthology.orchestra.run.smithi110.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-04-24T23:49:31.989 INFO:teuthology.orchestra.run.smithi110.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-04-24T23:49:32.014 INFO:teuthology.orchestra.run.smithi110.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-24T23:49:32.024 INFO:teuthology.orchestra.run.smithi188.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 411 kB/s | 206 kB 00:00 2024-04-24T23:49:32.048 INFO:teuthology.orchestra.run.smithi110.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-24T23:49:32.057 INFO:teuthology.orchestra.run.smithi188.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-24T23:49:32.098 INFO:teuthology.orchestra.run.smithi110.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-24T23:49:32.124 INFO:teuthology.orchestra.run.smithi110.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-24T23:49:32.165 INFO:teuthology.orchestra.run.smithi110.stdout:(65/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.2 MB/s | 3.0 MB 00:00 2024-04-24T23:49:32.191 INFO:teuthology.orchestra.run.smithi110.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 382 kB/s | 35 kB 00:00 2024-04-24T23:49:32.208 INFO:teuthology.orchestra.run.smithi188.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-04-24T23:49:32.325 INFO:teuthology.orchestra.run.smithi188.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-04-24T23:49:32.375 INFO:teuthology.orchestra.run.smithi188.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-04-24T23:49:32.391 INFO:teuthology.orchestra.run.smithi110.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-24T23:49:32.420 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-24T23:49:32.475 INFO:teuthology.orchestra.run.smithi188.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 87 kB/s | 23 kB 00:00 2024-04-24T23:49:32.550 INFO:teuthology.orchestra.run.smithi110.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-24T23:49:32.575 INFO:teuthology.orchestra.run.smithi110.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.9 MB/s | 911 kB 00:00 2024-04-24T23:49:32.592 INFO:teuthology.orchestra.run.smithi188.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-24T23:49:32.600 INFO:teuthology.orchestra.run.smithi110.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-24T23:49:32.659 INFO:teuthology.orchestra.run.smithi188.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 69 kB/s | 20 kB 00:00 2024-04-24T23:49:32.675 INFO:teuthology.orchestra.run.smithi110.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.6 MB/s | 4.7 MB 00:00 2024-04-24T23:49:32.742 INFO:teuthology.orchestra.run.smithi110.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-04-24T23:49:32.768 INFO:teuthology.orchestra.run.smithi110.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 951 kB/s | 87 kB 00:00 2024-04-24T23:49:32.826 INFO:teuthology.orchestra.run.smithi110.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T23:49:32.893 INFO:teuthology.orchestra.run.smithi110.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-24T23:49:32.934 INFO:teuthology.orchestra.run.smithi188.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-24T23:49:32.952 INFO:teuthology.orchestra.run.smithi110.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-04-24T23:49:33.010 INFO:teuthology.orchestra.run.smithi188.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 225 kB/s | 79 kB 00:00 2024-04-24T23:49:33.044 INFO:teuthology.orchestra.run.smithi110.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-24T23:49:33.069 INFO:teuthology.orchestra.run.smithi110.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-04-24T23:49:33.103 INFO:teuthology.orchestra.run.smithi110.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-24T23:49:33.128 INFO:teuthology.orchestra.run.smithi110.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-24T23:49:33.185 INFO:teuthology.orchestra.run.smithi188.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-04-24T23:49:33.203 INFO:teuthology.orchestra.run.smithi110.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-24T23:49:33.229 INFO:teuthology.orchestra.run.smithi110.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-24T23:49:33.262 INFO:teuthology.orchestra.run.smithi110.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-24T23:49:33.277 INFO:teuthology.orchestra.run.smithi188.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 168 kB/s | 58 kB 00:00 2024-04-24T23:49:33.287 INFO:teuthology.orchestra.run.smithi110.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T23:49:33.321 INFO:teuthology.orchestra.run.smithi110.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-04-24T23:49:33.347 INFO:teuthology.orchestra.run.smithi110.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T23:49:33.414 INFO:teuthology.orchestra.run.smithi110.stdout:(87/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-24T23:49:33.438 INFO:teuthology.orchestra.run.smithi110.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-24T23:49:33.461 INFO:teuthology.orchestra.run.smithi188.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 60 kB/s | 16 kB 00:00 2024-04-24T23:49:33.464 INFO:teuthology.orchestra.run.smithi110.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-24T23:49:33.489 INFO:teuthology.orchestra.run.smithi110.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-04-24T23:49:33.514 INFO:teuthology.orchestra.run.smithi110.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-04-24T23:49:33.519 INFO:teuthology.orchestra.run.smithi188.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-24T23:49:33.548 INFO:teuthology.orchestra.run.smithi110.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-04-24T23:49:33.573 INFO:teuthology.orchestra.run.smithi110.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-04-24T23:49:33.615 INFO:teuthology.orchestra.run.smithi110.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-24T23:49:33.628 INFO:teuthology.orchestra.run.smithi188.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 257 kB/s | 90 kB 00:00 2024-04-24T23:49:33.640 INFO:teuthology.orchestra.run.smithi110.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-04-24T23:49:33.666 INFO:teuthology.orchestra.run.smithi110.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-24T23:49:33.700 INFO:teuthology.orchestra.run.smithi110.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-24T23:49:33.720 INFO:teuthology.orchestra.run.smithi188.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 109 kB/s | 28 kB 00:00 2024-04-24T23:49:33.725 INFO:teuthology.orchestra.run.smithi110.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-04-24T23:49:33.750 INFO:teuthology.orchestra.run.smithi110.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T23:49:33.779 INFO:teuthology.orchestra.run.smithi188.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-04-24T23:49:33.786 INFO:teuthology.orchestra.run.smithi110.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T23:49:33.809 INFO:teuthology.orchestra.run.smithi110.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 760 kB/s | 45 kB 00:00 2024-04-24T23:49:33.834 INFO:teuthology.orchestra.run.smithi110.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 442 kB/s | 22 kB 00:00 2024-04-24T23:49:33.885 INFO:teuthology.orchestra.run.smithi110.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-24T23:49:34.045 INFO:teuthology.orchestra.run.smithi188.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-24T23:49:34.062 INFO:teuthology.orchestra.run.smithi188.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-04-24T23:49:34.088 INFO:teuthology.orchestra.run.smithi188.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 410 kB/s | 188 kB 00:00 2024-04-24T23:49:34.385 INFO:teuthology.orchestra.run.smithi110.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-24T23:49:34.396 INFO:teuthology.orchestra.run.smithi188.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-04-24T23:49:34.430 INFO:teuthology.orchestra.run.smithi188.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-04-24T23:49:34.486 INFO:teuthology.orchestra.run.smithi110.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 842 kB/s | 548 kB 00:00 2024-04-24T23:49:34.511 INFO:teuthology.orchestra.run.smithi110.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 494 kB/s | 309 kB 00:00 2024-04-24T23:49:34.571 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:49:34.572 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:49:34.580 INFO:teuthology.orchestra.run.smithi188.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 444 kB/s | 230 kB 00:00 2024-04-24T23:49:34.583 INFO:teuthology.orchestra.run.smithi173.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:49:34.584 INFO:teuthology.orchestra.run.smithi173.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:49:34.585 INFO:teuthology.orchestra.run.smithi173.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:49:34.585 INFO:teuthology.orchestra.run.smithi173.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:49:34.587 INFO:teuthology.orchestra.run.smithi173.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:49:34.589 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:49:34.590 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:49:34.611 INFO:teuthology.orchestra.run.smithi110.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-24T23:49:34.652 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-24T23:49:34.653 INFO:teuthology.orchestra.run.smithi110.stdout:(108/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:49:34.661 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:49:34.662 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:49:34.663 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2024-04-24T23:49:34.664 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:49:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:49:34.666 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:49:34.667 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:49:34.668 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:49:34.669 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:49:34.670 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:49:34.671 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:49:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:49:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout:Installing weak dependencies: 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:49:34.674 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-24T23:49:34.675 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-24T23:49:34.675 INFO:teuthology.orchestra.run.smithi173.stdout:Install 148 Packages 2024-04-24T23:49:34.675 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 4 Packages 2024-04-24T23:49:34.675 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:49:34.675 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 220 M 2024-04-24T23:49:34.675 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-24T23:49:34.688 INFO:teuthology.orchestra.run.smithi188.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 77 kB/s | 20 kB 00:00 2024-04-24T23:49:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-04-24T23:49:34.812 INFO:teuthology.orchestra.run.smithi110.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-24T23:49:34.837 INFO:teuthology.orchestra.run.smithi110.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 366 kB/s | 67 kB 00:00 2024-04-24T23:49:34.896 INFO:teuthology.orchestra.run.smithi110.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-24T23:49:34.988 INFO:teuthology.orchestra.run.smithi110.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 474 kB/s | 43 kB 00:00 2024-04-24T23:49:34.988 INFO:teuthology.orchestra.run.smithi188.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 721 kB/s | 427 kB 00:00 2024-04-24T23:49:35.013 INFO:teuthology.orchestra.run.smithi110.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-04-24T23:49:35.014 INFO:teuthology.orchestra.run.smithi188.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 442 kB/s | 192 kB 00:00 2024-04-24T23:49:35.080 INFO:teuthology.orchestra.run.smithi110.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-24T23:49:35.113 INFO:teuthology.orchestra.run.smithi110.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-24T23:49:35.155 INFO:teuthology.orchestra.run.smithi110.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-24T23:49:35.181 INFO:teuthology.orchestra.run.smithi110.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-24T23:49:35.214 INFO:teuthology.orchestra.run.smithi110.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-24T23:49:35.256 INFO:teuthology.orchestra.run.smithi110.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-04-24T23:49:35.281 INFO:teuthology.orchestra.run.smithi110.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-04-24T23:49:35.315 INFO:teuthology.orchestra.run.smithi110.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-24T23:49:35.373 INFO:teuthology.orchestra.run.smithi188.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-04-24T23:49:35.374 INFO:teuthology.orchestra.run.smithi110.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-24T23:49:35.407 INFO:teuthology.orchestra.run.smithi110.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-24T23:49:35.423 INFO:teuthology.orchestra.run.smithi188.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 7.8 MB/s | 3.4 MB 00:00 2024-04-24T23:49:35.438 INFO:teuthology.orchestra.run.smithi173.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-24T23:49:35.458 INFO:teuthology.orchestra.run.smithi110.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-24T23:49:35.525 INFO:teuthology.orchestra.run.smithi110.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-04-24T23:49:35.541 INFO:teuthology.orchestra.run.smithi110.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-04-24T23:49:35.548 INFO:teuthology.orchestra.run.smithi188.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-04-24T23:49:35.558 INFO:teuthology.orchestra.run.smithi110.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-24T23:49:35.617 INFO:teuthology.orchestra.run.smithi110.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-24T23:49:35.651 INFO:teuthology.orchestra.run.smithi110.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-24T23:49:35.676 INFO:teuthology.orchestra.run.smithi110.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 395 kB/s | 46 kB 00:00 2024-04-24T23:49:35.710 INFO:teuthology.orchestra.run.smithi110.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 853 kB/s | 79 kB 00:00 2024-04-24T23:49:35.752 INFO:teuthology.orchestra.run.smithi110.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-04-24T23:49:35.764 INFO:teuthology.orchestra.run.smithi173.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 2.7 MB/s | 890 kB 00:00 2024-04-24T23:49:35.785 INFO:teuthology.orchestra.run.smithi110.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-24T23:49:35.810 INFO:teuthology.orchestra.run.smithi110.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-24T23:49:35.844 INFO:teuthology.orchestra.run.smithi173.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.9 MB/s | 145 kB 00:00 2024-04-24T23:49:35.852 INFO:teuthology.orchestra.run.smithi110.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-04-24T23:49:35.878 INFO:teuthology.orchestra.run.smithi110.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-24T23:49:35.903 INFO:teuthology.orchestra.run.smithi110.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-04-24T23:49:35.953 INFO:teuthology.orchestra.run.smithi110.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-24T23:49:35.978 INFO:teuthology.orchestra.run.smithi110.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-04-24T23:49:36.004 INFO:teuthology.orchestra.run.smithi110.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-24T23:49:36.054 INFO:teuthology.orchestra.run.smithi110.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-24T23:49:36.080 INFO:teuthology.orchestra.run.smithi110.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-24T23:49:36.096 INFO:teuthology.orchestra.run.smithi110.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 974 kB/s | 90 kB 00:00 2024-04-24T23:49:36.155 INFO:teuthology.orchestra.run.smithi110.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-24T23:49:36.180 INFO:teuthology.orchestra.run.smithi110.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-24T23:49:36.205 INFO:teuthology.orchestra.run.smithi110.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-24T23:49:36.206 INFO:teuthology.orchestra.run.smithi188.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 1.3 MB 00:00 2024-04-24T23:49:36.231 INFO:teuthology.orchestra.run.smithi173.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 5.3 MB/s | 5.4 MB 00:01 2024-04-24T23:49:36.289 INFO:teuthology.orchestra.run.smithi173.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 5.2 MB/s | 2.4 MB 00:00 2024-04-24T23:49:36.448 INFO:teuthology.orchestra.run.smithi173.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-24T23:49:36.640 INFO:teuthology.orchestra.run.smithi188.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-04-24T23:49:36.643 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:49:36.643 INFO:teuthology.orchestra.run.smithi188.stdout:Total 12 MB/s | 220 MB 00:17 2024-04-24T23:49:36.939 INFO:teuthology.orchestra.run.smithi110.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 4.5 MB/s | 3.4 MB 00:00 2024-04-24T23:49:37.014 INFO:teuthology.orchestra.run.smithi110.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-04-24T23:49:37.090 INFO:teuthology.orchestra.run.smithi110.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-04-24T23:49:37.181 INFO:teuthology.orchestra.run.smithi173.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-04-24T23:49:37.198 INFO:teuthology.orchestra.run.smithi110.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-04-24T23:49:37.540 INFO:teuthology.orchestra.run.smithi110.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.3 MB/s | 3.1 MB 00:01 2024-04-24T23:49:37.543 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:49:37.543 INFO:teuthology.orchestra.run.smithi110.stdout:Total 13 MB/s | 220 MB 00:17 2024-04-24T23:49:37.747 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-04-24T23:49:37.895 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-04-24T23:49:37.895 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-04-24T23:49:38.182 INFO:teuthology.orchestra.run.smithi173.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 7.1 MB/s | 21 MB 00:02 2024-04-24T23:49:38.240 INFO:teuthology.orchestra.run.smithi173.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 432 kB/s | 25 kB 00:00 2024-04-24T23:49:38.390 INFO:teuthology.orchestra.run.smithi173.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 8.5 MB/s | 10 MB 00:01 2024-04-24T23:49:38.474 INFO:teuthology.orchestra.run.smithi173.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.6 MB/s | 222 kB 00:00 2024-04-24T23:49:38.641 INFO:teuthology.orchestra.run.smithi173.stdout:(12/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 7.7 MB/s | 17 MB 00:02 2024-04-24T23:49:38.654 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-24T23:49:38.666 INFO:teuthology.orchestra.run.smithi173.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 141 kB/s | 27 kB 00:00 2024-04-24T23:49:38.700 INFO:teuthology.orchestra.run.smithi173.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 544 kB/s | 32 kB 00:00 2024-04-24T23:49:38.758 INFO:teuthology.orchestra.run.smithi173.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.7 MB/s | 163 kB 00:00 2024-04-24T23:49:38.783 INFO:teuthology.orchestra.run.smithi173.stdout:(16/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 6.2 MB/s | 745 kB 00:00 2024-04-24T23:49:38.799 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-24T23:49:38.799 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-24T23:49:38.826 INFO:teuthology.orchestra.run.smithi173.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.8 MB/s | 127 kB 00:00 2024-04-24T23:49:38.859 INFO:teuthology.orchestra.run.smithi173.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 6.5 MB/s | 501 kB 00:00 2024-04-24T23:49:38.926 INFO:teuthology.orchestra.run.smithi173.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 677 kB/s | 45 kB 00:00 2024-04-24T23:49:39.010 INFO:teuthology.orchestra.run.smithi173.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.6 MB/s | 133 kB 00:00 2024-04-24T23:49:39.102 INFO:teuthology.orchestra.run.smithi173.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.7 MB/s | 162 kB 00:00 2024-04-24T23:49:39.185 INFO:teuthology.orchestra.run.smithi173.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.8 MB/s | 321 kB 00:00 2024-04-24T23:49:39.269 INFO:teuthology.orchestra.run.smithi173.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 299 kB 00:00 2024-04-24T23:49:39.352 INFO:teuthology.orchestra.run.smithi173.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T23:49:39.428 INFO:teuthology.orchestra.run.smithi173.stdout:(25/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 8.8 MB/s | 5.3 MB 00:00 2024-04-24T23:49:39.453 INFO:teuthology.orchestra.run.smithi173.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 851 kB/s | 85 kB 00:00 2024-04-24T23:49:39.520 INFO:teuthology.orchestra.run.smithi173.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-24T23:49:39.604 INFO:teuthology.orchestra.run.smithi173.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 382 kB/s | 32 kB 00:00 2024-04-24T23:49:39.636 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-04-24T23:49:39.636 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-04-24T23:49:39.687 INFO:teuthology.orchestra.run.smithi173.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.7 MB/s | 141 kB 00:00 2024-04-24T23:49:39.829 INFO:teuthology.orchestra.run.smithi173.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 7.7 MB/s | 3.1 MB 00:00 2024-04-24T23:49:40.179 INFO:teuthology.orchestra.run.smithi173.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 7.1 MB/s | 3.5 MB 00:00 2024-04-24T23:49:40.313 INFO:teuthology.orchestra.run.smithi173.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.8 MB/s | 249 kB 00:00 2024-04-24T23:49:40.406 INFO:teuthology.orchestra.run.smithi173.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 536 kB/s | 49 kB 00:00 2024-04-24T23:49:40.489 INFO:teuthology.orchestra.run.smithi173.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 184 kB/s | 15 kB 00:00 2024-04-24T23:49:40.556 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-24T23:49:40.558 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-24T23:49:40.598 INFO:teuthology.orchestra.run.smithi173.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.4 MB/s | 264 kB 00:00 2024-04-24T23:49:40.741 INFO:teuthology.orchestra.run.smithi173.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 5.3 MB/s | 762 kB 00:00 2024-04-24T23:49:40.882 INFO:teuthology.orchestra.run.smithi173.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-04-24T23:49:40.907 INFO:teuthology.orchestra.run.smithi173.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 109 kB/s | 18 kB 00:00 2024-04-24T23:49:40.974 INFO:teuthology.orchestra.run.smithi173.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 875 kB/s | 80 kB 00:00 2024-04-24T23:49:41.024 INFO:teuthology.orchestra.run.smithi173.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-24T23:49:41.066 INFO:teuthology.orchestra.run.smithi173.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-24T23:49:41.118 INFO:teuthology.orchestra.run.smithi173.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-24T23:49:41.183 INFO:teuthology.orchestra.run.smithi173.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-24T23:49:41.225 INFO:teuthology.orchestra.run.smithi173.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-04-24T23:49:41.267 INFO:teuthology.orchestra.run.smithi173.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-24T23:49:41.309 INFO:teuthology.orchestra.run.smithi173.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-24T23:49:41.359 INFO:teuthology.orchestra.run.smithi173.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-24T23:49:41.426 INFO:teuthology.orchestra.run.smithi173.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-04-24T23:49:41.477 INFO:teuthology.orchestra.run.smithi173.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-04-24T23:49:41.519 INFO:teuthology.orchestra.run.smithi173.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-24T23:49:41.544 INFO:teuthology.orchestra.run.smithi173.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 320 kB/s | 21 kB 00:00 2024-04-24T23:49:41.602 INFO:teuthology.orchestra.run.smithi173.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-04-24T23:49:41.652 INFO:teuthology.orchestra.run.smithi173.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-24T23:49:41.711 INFO:teuthology.orchestra.run.smithi173.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-24T23:49:41.753 INFO:teuthology.orchestra.run.smithi173.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-24T23:49:41.854 INFO:teuthology.orchestra.run.smithi173.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-24T23:49:41.921 INFO:teuthology.orchestra.run.smithi173.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 679 kB/s | 45 kB 00:00 2024-04-24T23:49:42.437 INFO:teuthology.orchestra.run.smithi173.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 662 kB 00:00 2024-04-24T23:49:42.612 INFO:teuthology.orchestra.run.smithi173.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 3.3 MB/s | 582 kB 00:00 2024-04-24T23:49:42.665 INFO:teuthology.orchestra.run.smithi173.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 489 kB/s | 25 kB 00:00 2024-04-24T23:49:42.732 INFO:teuthology.orchestra.run.smithi173.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-04-24T23:49:42.824 INFO:teuthology.orchestra.run.smithi173.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-24T23:49:43.241 INFO:teuthology.orchestra.run.smithi173.stdout:(63/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 9.7 MB/s | 48 MB 00:05 2024-04-24T23:49:43.299 INFO:teuthology.orchestra.run.smithi173.stdout:(64/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.8 MB/s | 3.0 MB 00:01 2024-04-24T23:49:43.341 INFO:teuthology.orchestra.run.smithi173.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 353 kB/s | 35 kB 00:00 2024-04-24T23:49:43.367 INFO:teuthology.orchestra.run.smithi173.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 347 kB/s | 188 kB 00:00 2024-04-24T23:49:43.550 INFO:teuthology.orchestra.run.smithi173.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-04-24T23:49:43.771 INFO:teuthology.orchestra.run.smithi173.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 911 kB 00:00 2024-04-24T23:49:43.884 INFO:teuthology.orchestra.run.smithi173.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-04-24T23:49:43.918 INFO:teuthology.orchestra.run.smithi173.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 910 kB/s | 133 kB 00:00 2024-04-24T23:49:44.043 INFO:teuthology.orchestra.run.smithi173.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.4 MB/s | 4.7 MB 00:00 2024-04-24T23:49:44.127 INFO:teuthology.orchestra.run.smithi173.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-04-24T23:49:44.177 INFO:teuthology.orchestra.run.smithi173.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-04-24T23:49:44.269 INFO:teuthology.orchestra.run.smithi173.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-24T23:49:44.361 INFO:teuthology.orchestra.run.smithi173.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-24T23:49:44.436 INFO:teuthology.orchestra.run.smithi173.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 343 kB/s | 26 kB 00:00 2024-04-24T23:49:44.611 INFO:teuthology.orchestra.run.smithi173.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 109 kB/s | 19 kB 00:00 2024-04-24T23:49:44.694 INFO:teuthology.orchestra.run.smithi173.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-24T23:49:44.770 INFO:teuthology.orchestra.run.smithi173.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-04-24T23:49:44.978 INFO:teuthology.orchestra.run.smithi173.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-24T23:49:45.097 INFO:teuthology.orchestra.run.smithi173.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.2 MB/s | 6.0 MB 00:00 2024-04-24T23:49:45.120 INFO:teuthology.orchestra.run.smithi173.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-24T23:49:45.180 INFO:teuthology.orchestra.run.smithi173.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T23:49:45.205 INFO:teuthology.orchestra.run.smithi173.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-24T23:49:45.272 INFO:teuthology.orchestra.run.smithi173.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 459 kB/s | 42 kB 00:00 2024-04-24T23:49:45.363 INFO:teuthology.orchestra.run.smithi173.stdout:(86/152): postgresql-server-13.14-1.el9.x86_64. 3.9 MB/s | 5.8 MB 00:01 2024-04-24T23:49:45.388 INFO:teuthology.orchestra.run.smithi173.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-04-24T23:49:45.422 INFO:teuthology.orchestra.run.smithi173.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-24T23:49:45.473 INFO:teuthology.orchestra.run.smithi173.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-24T23:49:45.498 INFO:teuthology.orchestra.run.smithi173.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-04-24T23:49:45.508 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-04-24T23:49:45.524 INFO:teuthology.orchestra.run.smithi173.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 533 kB/s | 54 kB 00:00 2024-04-24T23:49:45.573 INFO:teuthology.orchestra.run.smithi173.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-24T23:49:45.599 INFO:teuthology.orchestra.run.smithi173.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-04-24T23:49:45.610 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:49:45.649 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:49:45.657 INFO:teuthology.orchestra.run.smithi173.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-04-24T23:49:45.710 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:49:45.724 INFO:teuthology.orchestra.run.smithi173.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-04-24T23:49:45.749 INFO:teuthology.orchestra.run.smithi173.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 516 kB/s | 47 kB 00:00 2024-04-24T23:49:45.752 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:49:45.778 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:49:45.816 INFO:teuthology.orchestra.run.smithi173.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-04-24T23:49:45.823 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:49:45.842 INFO:teuthology.orchestra.run.smithi173.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.5 MB/s | 305 kB 00:00 2024-04-24T23:49:45.901 INFO:teuthology.orchestra.run.smithi173.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T23:49:45.926 INFO:teuthology.orchestra.run.smithi173.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 863 kB/s | 94 kB 00:00 2024-04-24T23:49:45.976 INFO:teuthology.orchestra.run.smithi173.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-24T23:49:46.001 INFO:teuthology.orchestra.run.smithi173.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 446 kB/s | 45 kB 00:00 2024-04-24T23:49:46.110 INFO:teuthology.orchestra.run.smithi173.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-04-24T23:49:46.135 INFO:teuthology.orchestra.run.smithi173.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-04-24T23:49:46.151 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:49:46.169 INFO:teuthology.orchestra.run.smithi173.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 584 kB/s | 19 kB 00:00 2024-04-24T23:49:46.236 INFO:teuthology.orchestra.run.smithi173.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-04-24T23:49:46.254 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:49:46.269 INFO:teuthology.orchestra.run.smithi173.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 778 kB/s | 26 kB 00:00 2024-04-24T23:49:46.328 INFO:teuthology.orchestra.run.smithi173.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-24T23:49:46.353 INFO:teuthology.orchestra.run.smithi173.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-04-24T23:49:46.367 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:49:46.370 INFO:teuthology.orchestra.run.smithi173.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-24T23:49:46.396 INFO:teuthology.orchestra.run.smithi173.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-04-24T23:49:46.397 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:49:46.429 INFO:teuthology.orchestra.run.smithi173.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-04-24T23:49:46.454 INFO:teuthology.orchestra.run.smithi173.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-04-24T23:49:46.472 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:49:46.480 INFO:teuthology.orchestra.run.smithi173.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-24T23:49:46.481 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-24T23:49:46.499 INFO:teuthology.orchestra.run.smithi173.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 770 kB/s | 32 kB 00:00 2024-04-24T23:49:46.501 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:49:46.513 INFO:teuthology.orchestra.run.smithi173.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 448 kB/s | 15 kB 00:00 2024-04-24T23:49:46.539 INFO:teuthology.orchestra.run.smithi173.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-04-24T23:49:46.564 INFO:teuthology.orchestra.run.smithi173.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-24T23:49:46.569 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:49:46.589 INFO:teuthology.orchestra.run.smithi173.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-04-24T23:49:46.608 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:49:46.615 INFO:teuthology.orchestra.run.smithi173.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 698 kB/s | 35 kB 00:00 2024-04-24T23:49:46.648 INFO:teuthology.orchestra.run.smithi173.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-24T23:49:46.655 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:49:46.661 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:49:46.695 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:49:46.707 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:49:46.720 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:49:46.735 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:49:46.763 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:49:46.807 INFO:teuthology.orchestra.run.smithi173.stdout:(122/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-24T23:49:46.821 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:49:46.832 INFO:teuthology.orchestra.run.smithi173.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 787 kB/s | 144 kB 00:00 2024-04-24T23:49:46.845 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:49:46.883 INFO:teuthology.orchestra.run.smithi173.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-04-24T23:49:46.890 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:49:46.907 INFO:teuthology.orchestra.run.smithi173.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-24T23:49:46.912 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:49:46.933 INFO:teuthology.orchestra.run.smithi173.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-24T23:49:46.935 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:49:46.958 INFO:teuthology.orchestra.run.smithi173.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-24T23:49:46.983 INFO:teuthology.orchestra.run.smithi173.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-24T23:49:47.009 INFO:teuthology.orchestra.run.smithi173.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-24T23:49:47.034 INFO:teuthology.orchestra.run.smithi173.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 922 kB/s | 46 kB 00:00 2024-04-24T23:49:47.068 INFO:teuthology.orchestra.run.smithi173.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-24T23:49:47.093 INFO:teuthology.orchestra.run.smithi173.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-04-24T23:49:47.102 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:49:47.112 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:49:47.118 INFO:teuthology.orchestra.run.smithi173.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-04-24T23:49:47.143 INFO:teuthology.orchestra.run.smithi173.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-24T23:49:47.144 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:49:47.160 INFO:teuthology.orchestra.run.smithi173.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-04-24T23:49:47.185 INFO:teuthology.orchestra.run.smithi173.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 679 kB/s | 28 kB 00:00 2024-04-24T23:49:47.211 INFO:teuthology.orchestra.run.smithi173.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-04-24T23:49:47.229 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:49:47.236 INFO:teuthology.orchestra.run.smithi173.stdout:(138/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-24T23:49:47.251 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:49:47.261 INFO:teuthology.orchestra.run.smithi173.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-24T23:49:47.286 INFO:teuthology.orchestra.run.smithi173.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-24T23:49:47.312 INFO:teuthology.orchestra.run.smithi173.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-24T23:49:47.342 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:49:47.351 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:49:47.357 INFO:teuthology.orchestra.run.smithi173.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 536 kB/s | 51 kB 00:00 2024-04-24T23:49:47.380 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:49:47.386 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:49:47.387 INFO:teuthology.orchestra.run.smithi173.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T23:49:47.413 INFO:teuthology.orchestra.run.smithi173.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-24T23:49:47.429 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:49:47.438 INFO:teuthology.orchestra.run.smithi173.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-24T23:49:47.463 INFO:teuthology.orchestra.run.smithi173.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-24T23:49:47.472 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:49:47.488 INFO:teuthology.orchestra.run.smithi173.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.2 MB/s | 427 kB 00:00 2024-04-24T23:49:47.501 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:49:47.504 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:49:47.544 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:49:47.555 INFO:teuthology.orchestra.run.smithi173.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-24T23:49:47.588 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:49:47.621 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:49:47.649 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:49:47.650 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:49:47.672 INFO:teuthology.orchestra.run.smithi173.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-24T23:49:47.715 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:49:47.719 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:49:47.743 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:49:47.772 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:49:47.801 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:49:47.831 INFO:teuthology.orchestra.run.smithi173.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-04-24T23:49:47.836 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:49:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:49:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:49:47.914 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:49:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:49:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:49:47.948 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:49:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:49:47.972 INFO:teuthology.orchestra.run.smithi173.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 6.7 MB/s | 3.4 MB 00:00 2024-04-24T23:49:47.991 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:49:48.020 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:49:48.023 INFO:teuthology.orchestra.run.smithi173.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-04-24T23:49:48.025 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:49:48.026 INFO:teuthology.orchestra.run.smithi173.stdout:Total 16 MB/s | 220 MB 00:13 2024-04-24T23:49:48.093 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:49:48.134 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:49:48.153 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:49:48.195 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:49:48.211 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:49:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:49:48.358 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:49:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:49:48.414 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:49:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:49:48.462 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:49:48.526 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:49:48.570 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:49:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:49:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:49:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:49:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:49:48.803 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:49:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:49:48.878 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:49:48.954 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:49:48.972 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:49:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:49:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:49:49.061 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:49:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:49:49.096 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:49:49.124 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-24T23:49:49.138 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:49:49.139 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:49:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:49:49.192 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:49:49.219 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:49:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:49:49.269 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-24T23:49:49.269 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-24T23:49:49.393 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:49:49.407 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:49:49.455 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:49:49.461 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:49:49.511 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:49:49.585 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:49:49.622 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:49:49.659 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:49:49.706 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:49:49.741 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:49:49.785 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:49:49.827 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:49:49.969 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:49:50.011 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:49:50.015 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:49:50.046 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:49:50.087 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:49:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:49:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:49:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:49:50.184 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:49:50.209 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:49:50.220 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:49:50.251 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:49:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:49:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:49:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:49:50.387 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:49:50.407 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:49:50.462 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:49:50.516 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:49:50.517 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:49:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:49:50.580 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:49:50.590 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:49:50.621 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:49:50.622 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:49:50.657 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:49:50.662 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:49:50.694 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:49:50.704 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:49:50.723 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:49:50.740 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:49:50.758 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:49:50.783 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:49:50.791 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:49:50.825 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:49:50.833 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:49:50.867 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:49:50.956 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:49:50.960 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:49:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:49:51.034 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-24T23:49:51.035 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-24T23:49:51.036 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:49:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:49:51.078 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:49:51.137 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:49:51.171 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:49:51.185 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:49:51.269 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:49:51.294 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:49:51.309 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:49:51.335 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:49:51.356 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:49:51.359 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:49:51.433 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:49:51.560 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:49:51.609 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:49:51.637 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:49:51.671 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:49:51.710 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:49:51.741 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:49:51.771 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:49:51.806 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:49:51.839 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:49:51.889 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:49:51.924 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:49:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:49:52.161 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:49:52.195 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:49:52.216 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:49:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:49:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:49:52.296 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:49:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:49:52.345 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:49:52.434 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:49:52.439 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:49:52.480 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:49:52.495 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:49:52.656 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:49:52.686 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:49:52.764 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:49:53.211 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:49:53.259 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:49:53.293 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:49:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:49:53.447 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:49:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:49:53.554 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:49:53.570 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:49:53.603 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:49:53.780 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:49:53.809 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:49:53.887 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:49:54.526 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:49:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:49:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:49:54.941 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:49:54.969 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:49:55.002 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:49:55.312 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:49:55.365 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:49:55.439 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:49:55.495 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:49:55.531 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:49:55.853 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:49:55.939 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:49:55.967 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:49:56.000 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:49:56.001 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:49:56.034 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:49:56.106 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:49:56.136 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:49:56.329 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:49:56.371 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:49:56.462 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:49:56.509 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:49:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:49:56.959 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-24T23:49:57.026 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:49:57.044 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:49:57.074 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:49:57.081 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:49:57.135 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:49:57.169 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:49:57.194 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:49:57.237 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:49:57.381 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:49:57.410 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:49:57.576 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:49:57.680 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:49:57.810 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:49:57.838 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:49:57.939 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:49:58.002 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:49:58.150 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:49:58.207 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:49:58.234 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:49:58.337 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:49:58.365 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:49:58.427 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:49:58.431 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:49:58.453 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:49:58.466 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:49:58.473 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:49:58.620 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:49:58.652 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:49:58.759 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:49:58.846 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:49:58.885 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:49:58.919 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:49:58.984 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:49:59.008 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:49:59.027 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:49:59.053 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:49:59.070 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:49:59.102 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:49:59.130 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:49:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:49:59.148 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:49:59.148 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:49:59.148 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:49:59.188 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:49:59.191 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:49:59.244 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:49:59.275 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:49:59.310 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:49:59.385 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:49:59.421 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:49:59.464 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:49:59.493 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:49:59.563 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:49:59.616 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:49:59.619 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:49:59.648 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:49:59.690 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:49:59.821 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:49:59.877 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:50:00.205 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:50:00.252 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:50:00.344 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:50:00.344 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:50:00.344 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:50:00.344 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:00.378 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:50:00.400 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:50:00.481 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:50:00.510 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:50:00.544 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:50:00.582 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:50:00.623 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:50:00.772 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:50:00.818 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:50:00.873 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:50:00.946 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:50:00.977 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:50:01.014 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:50:01.060 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:50:01.096 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:50:01.139 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:50:01.181 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:50:01.318 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:50:01.370 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:50:01.400 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:50:01.442 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:50:01.535 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:50:01.567 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:50:01.607 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:50:01.632 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:50:01.657 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:50:01.731 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:50:01.859 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:50:01.897 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:50:01.926 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:50:01.960 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:50:01.999 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:50:02.030 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:50:02.085 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:50:02.253 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:50:02.404 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:50:02.446 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:50:02.479 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:50:02.567 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:50:02.618 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:50:02.649 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:50:02.696 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:50:02.794 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:50:02.851 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:50:03.574 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:50:03.623 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:50:03.656 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:50:03.756 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:50:03.814 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:50:03.901 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:50:03.933 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:50:04.111 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:50:04.373 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:50:04.456 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:50:05.093 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:50:05.212 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:50:06.424 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:50:06.517 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:50:06.545 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:50:06.579 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:50:06.888 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:50:06.933 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:50:07.023 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:50:07.054 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:50:07.090 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:50:07.559 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:50:07.586 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:50:07.640 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:50:07.662 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:50:09.907 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:50:09.934 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:50:10.480 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:50:10.520 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:50:10.604 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:50:10.604 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:50:10.604 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:50:10.604 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:10.638 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:50:10.662 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:50:10.662 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-04-24T23:50:10.662 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-04-24T23:50:10.662 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-04-24T23:50:10.662 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-04-24T23:50:10.663 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-04-24T23:50:10.663 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-04-24T23:50:10.663 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-04-24T23:50:10.663 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:10.850 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:50:10.865 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:50:10.865 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:10.866 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:50:10.866 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:11.729 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:50:11.729 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-04-24T23:50:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-04-24T23:50:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-04-24T23:50:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-04-24T23:50:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-04-24T23:50:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-04-24T23:50:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-04-24T23:50:11.730 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:50:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:50:11.906 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:11.906 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:50:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:12.312 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:50:12.337 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:50:12.337 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:12.338 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:50:12.338 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:50:12.338 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:50:12.338 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:12.562 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:50:12.584 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:50:12.584 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:12.584 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:50:12.584 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:50:12.584 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:50:12.585 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:13.029 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:50:13.052 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:50:13.053 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:13.053 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:50:13.053 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:50:13.053 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:50:13.053 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:13.160 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:50:13.217 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:50:13.273 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:50:13.394 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:50:13.419 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:50:13.420 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:13.420 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:50:13.420 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:50:13.420 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:50:13.420 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:13.465 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:50:13.523 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:50:13.610 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:50:13.645 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:50:13.668 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:50:13.668 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:13.668 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:50:13.668 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:50:13.668 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:50:13.668 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:13.698 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:50:13.840 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:50:13.881 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:50:13.913 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:50:13.949 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:50:14.104 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:50:14.127 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:50:14.127 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:14.127 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:50:14.127 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:50:14.127 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:50:14.127 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:14.234 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:50:14.287 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:50:14.338 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:50:14.348 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:50:14.380 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:50:14.412 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:50:14.478 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:50:14.504 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:50:14.522 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:50:14.572 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:50:14.659 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:50:14.754 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:50:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:50:14.939 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:50:14.970 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:50:14.998 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:50:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:50:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:50:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:50:15.544 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:50:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:50:15.604 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:50:15.606 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:50:16.565 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:50:16.590 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:50:16.633 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:50:16.680 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:50:16.693 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:50:16.851 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:50:16.875 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:50:16.898 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:50:16.898 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:16.898 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:50:16.898 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:50:16.898 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:50:16.898 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:16.950 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:50:16.961 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:50:17.051 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:50:17.099 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:50:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:50:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:50:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:50:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:50:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:50:17.994 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:50:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:50:17.998 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:17.998 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:50:17.998 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:50:17.998 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:50:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:18.028 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:50:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:50:18.051 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:50:18.051 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:18.051 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:50:18.051 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:50:18.051 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:50:18.051 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:50:18.095 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:50:18.117 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:50:18.117 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:18.117 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:50:18.118 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:18.158 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:50:18.166 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:50:18.184 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:50:18.185 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:18.185 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:50:18.185 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:50:18.185 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:50:18.185 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:18.206 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:50:18.480 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:50:18.503 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:50:18.503 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:18.503 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:50:18.504 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:50:18.504 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:50:18.504 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:50:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:50:19.159 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:50:19.159 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:19.159 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:50:19.160 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:50:19.160 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:50:19.160 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:50:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:50:19.225 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:19.226 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:50:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:19.271 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:50:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:50:19.293 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:19.294 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:50:19.294 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:50:19.294 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:50:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:50:19.618 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:50:19.619 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:19.619 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:50:19.619 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:50:19.619 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:50:19.619 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:22.076 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:50:22.076 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2024-04-24T23:50:22.077 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2024-04-24T23:50:22.077 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2024-04-24T23:50:22.077 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2024-04-24T23:50:22.077 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2024-04-24T23:50:22.077 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2024-04-24T23:50:22.077 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2024-04-24T23:50:22.077 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:22.254 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:50:22.267 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:50:22.267 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:22.267 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:50:22.267 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:22.967 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:50:23.008 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:50:23.049 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:50:23.081 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:50:23.108 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:50:23.302 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:50:23.355 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:50:23.385 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:50:23.745 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:50:23.771 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:50:23.772 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:23.772 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:50:23.772 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:50:23.772 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:50:23.772 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:23.930 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:23.966 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:23.985 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:23.996 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:50:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:50:24.018 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:24.018 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:50:24.018 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:50:24.018 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:50:24.018 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:50:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:50:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:50:24.231 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:50:24.259 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:50:24.444 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:50:24.454 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:50:24.477 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:50:24.477 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:24.477 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:50:24.477 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:50:24.477 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:50:24.477 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:24.488 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:50:24.517 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:50:24.585 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:50:24.634 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:50:24.688 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:50:24.842 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:50:24.876 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:50:24.921 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:50:24.922 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:50:24.962 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:50:24.991 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:50:25.046 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:50:25.075 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:50:25.075 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:50:25.080 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:50:25.140 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:25.159 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:50:25.174 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:25.192 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:50:25.192 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:50:25.194 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:25.219 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:50:25.263 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:50:25.270 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:50:25.296 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:50:25.323 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:50:25.341 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:25.680 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:50:25.705 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:50:25.736 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:50:25.798 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:50:25.828 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:50:26.021 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:50:26.112 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:50:26.154 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:50:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:50:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:50:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:50:26.334 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:50:26.367 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:50:26.367 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:50:26.449 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:50:26.515 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:26.849 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:50:26.898 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:50:27.863 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:50:27.896 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:50:28.000 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:50:28.168 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:50:28.199 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:50:28.222 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:50:28.222 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:28.222 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:50:28.222 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:50:28.222 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:50:28.222 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:28.266 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:50:28.277 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:50:28.375 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:50:28.423 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:50:29.319 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:50:29.343 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:50:29.366 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:50:29.366 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:29.366 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:50:29.366 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:50:29.366 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:50:29.367 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:29.410 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:50:29.432 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:50:29.432 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:29.433 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:50:29.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:29.478 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:50:29.500 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:50:29.500 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:29.500 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:50:29.500 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:50:29.500 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:50:29.501 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:29.803 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:50:29.826 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:50:29.826 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:50:29.827 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:50:29.827 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:50:29.827 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:50:29.827 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:50:31.284 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:50:31.285 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:50:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:50:31.287 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:50:31.288 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:50:31.289 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:50:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:50:31.291 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:50:31.292 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:50:31.293 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:50:31.294 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:50:31.295 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:50:32.393 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:50:32.394 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:50:32.395 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:50:32.396 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:50:32.396 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:50:32.407 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:50:32.408 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:50:32.409 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:50:32.410 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:50:32.410 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:50:32.410 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:50:32.410 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:50:32.410 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:50:32.410 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:50:32.410 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:50:32.410 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:50:32.411 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:50:32.412 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:50:32.413 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:50:32.414 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:50:32.415 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:50:32.416 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:50:32.417 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:50:32.418 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:50:32.419 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:50:32.419 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:50:33.457 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.458 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:50:33.459 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:50:33.460 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:50:33.461 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:50:33.462 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:50:33.463 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:50:33.464 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:50:33.465 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:50:33.466 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:50:33.467 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:50:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:50:33.468 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:50:33.468 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-24T23:50:33.818 DEBUG:teuthology.parallel:result is None 2024-04-24T23:50:34.349 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:50:34.390 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:50:34.431 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:50:34.463 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:50:34.490 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:50:34.505 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:34.505 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:34.505 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-04-24T23:50:34.505 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.505 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.505 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:34.505 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:34.505 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.506 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.507 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.508 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:50:34.509 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:50:34.510 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:50:34.511 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:50:34.512 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:50:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:50:34.514 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:50:34.515 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:50:34.516 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:50:34.516 INFO:teuthology.orchestra.run.smithi110.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:50:34.516 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:50:34.516 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:50:34.516 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:50:34.516 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:50:34.516 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-24T23:50:34.673 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:50:34.719 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:50:34.749 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:50:34.955 DEBUG:teuthology.parallel:result is None 2024-04-24T23:50:35.287 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:35.329 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:35.349 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:50:36.182 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:50:36.267 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:50:36.300 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:50:36.390 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:50:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:50:36.430 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:50:36.514 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:50:36.547 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:50:36.547 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:50:36.641 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:50:36.712 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:42.671 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:42.671 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:50:42.672 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:50:42.673 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:50:42.674 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:50:42.675 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:50:42.676 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:50:42.677 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:50:42.678 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:50:42.679 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:50:42.681 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:50:42.682 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:50:42.683 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:50:42.683 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:50:42.683 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:50:42.683 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:50:42.684 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:50:42.685 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:50:44.723 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:50:44.723 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:44.723 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2024-04-24T23:50:44.723 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.723 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.723 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:44.723 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:44.723 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.724 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.725 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:50:44.726 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:50:44.727 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:50:44.728 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.729 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:50:44.730 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:50:44.731 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:50:44.733 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:50:44.733 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:50:44.733 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.734 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:50:44.735 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:50:44.736 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:50:44.737 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:50:44.738 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:50:44.738 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-24T23:50:44.738 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-24T23:50:45.145 DEBUG:teuthology.parallel:result is None 2024-04-24T23:50:45.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:50:45.292 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:50:45.319 INFO:teuthology.orchestra.run.smithi110.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:50:45.320 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:50:45.320 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:50:45.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:50:45.465 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:50:45.485 INFO:teuthology.orchestra.run.smithi173.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:50:45.486 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:50:45.486 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:50:45.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:50:45.631 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:50:45.656 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:50:45.657 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:50:45.657 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:50:45.659 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:50:45.659 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:50:45.659 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:50:45.691 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:50:45.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:50:45.719 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:50:45.719 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:50:45.749 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:50:45.750 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:50:45.750 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:50:45.777 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:50:45.844 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:50:45.844 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:50:45.868 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:50:45.932 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:50:45.933 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:50:45.961 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:50:46.029 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:50:46.029 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:50:46.029 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:50:46.055 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:50:46.121 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:50:46.121 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:50:46.146 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:50:46.210 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:50:46.210 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:50:46.238 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:50:46.305 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:50:46.305 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:50:46.306 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:50:46.330 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:50:46.394 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:50:46.394 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:50:46.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:50:46.481 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:50:46.481 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:50:46.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:50:46.576 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T23:50:46.678 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-24T23:50:46.678 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:50:46.679 INFO:tasks.cephadm:Cluster fsid is 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:50:46.679 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T23:50:46.679 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.110', 'mon.b': '172.21.15.173', 'mon.c': '172.21.15.188'} 2024-04-24T23:50:46.679 INFO:tasks.cephadm:First mon is mon.a on smithi110 2024-04-24T23:50:46.679 INFO:tasks.cephadm:First mgr is x 2024-04-24T23:50:46.679 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T23:50:46.679 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-04-24T23:50:46.704 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2024-04-24T23:50:46.728 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname $(hostname -s) 2024-04-24T23:50:46.755 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T23:50:46.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:50:46.974 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T23:50:47.194 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:50:47.196 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-24T23:50:47.196 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:47.574 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:50 /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:47.574 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:48.112 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:50 /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:48.112 DEBUG:teuthology.orchestra.run.smithi188:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:48.490 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:50 /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:48.490 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:48.509 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:48.529 DEBUG:teuthology.orchestra.run.smithi188:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:50:48.560 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-24T23:50:48.561 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:50:48.563 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:50:48.571 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:50:50.104 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:50:50.185 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:50:50.257 INFO:teuthology.orchestra.run.smithi188.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:51:23.511 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-24T23:51:23.511 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:51:23.511 INFO:teuthology.orchestra.run.smithi188.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:51:23.512 INFO:teuthology.orchestra.run.smithi188.stdout: "repo_digests": [ 2024-04-24T23:51:23.512 INFO:teuthology.orchestra.run.smithi188.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:51:23.512 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-04-24T23:51:23.512 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-24T23:51:40.739 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-24T23:51:40.739 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:51:40.739 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:51:40.739 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2024-04-24T23:51:40.739 INFO:teuthology.orchestra.run.smithi173.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:51:40.739 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-04-24T23:51:40.739 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-24T23:51:44.388 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-24T23:51:44.388 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:51:44.388 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:51:44.388 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-04-24T23:51:44.388 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:51:44.388 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-24T23:51:44.388 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-24T23:51:44.413 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-04-24T23:51:44.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2024-04-24T23:51:44.485 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph 2024-04-24T23:51:44.516 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-04-24T23:51:44.540 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2024-04-24T23:51:44.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /etc/ceph 2024-04-24T23:51:44.593 INFO:tasks.cephadm:Writing seed config... 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T23:51:44.595 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T23:51:44.596 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T23:51:44.597 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T23:51:44.598 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T23:51:44.598 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T23:51:44.598 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-24T23:51:44.598 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:51:44.599 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T23:51:44.640 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 787fa75a-0295-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T23:51:44.641 DEBUG:teuthology.orchestra.run.smithi110:mon.a> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T23:51:44.683 DEBUG:teuthology.orchestra.run.smithi110:mgr.x> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T23:51:44.726 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T23:51:44.727 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 787fa75a-0295-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.110 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:51:44.915 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:51:44.915 INFO:teuthology.orchestra.run.smithi110.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '787fa75a-0295-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.110', '--skip-admin-label'] 2024-04-24T23:51:44.915 INFO:teuthology.orchestra.run.smithi110.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T23:51:44.915 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying podman|docker is present... 2024-04-24T23:51:44.942 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:51:44.942 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying lvm2 is present... 2024-04-24T23:51:44.942 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying time synchronization is in place... 2024-04-24T23:51:44.947 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:51:44.948 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:51:44.954 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:51:44.954 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-24T23:51:44.960 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout enabled 2024-04-24T23:51:44.966 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout active 2024-04-24T23:51:44.966 INFO:teuthology.orchestra.run.smithi110.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:51:44.966 INFO:teuthology.orchestra.run.smithi110.stdout:Repeating the final host check... 2024-04-24T23:51:44.989 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:51:44.989 INFO:teuthology.orchestra.run.smithi110.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T23:51:44.989 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl is present 2024-04-24T23:51:44.989 INFO:teuthology.orchestra.run.smithi110.stdout:lvcreate is present 2024-04-24T23:51:44.995 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:51:44.995 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:51:45.001 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:51:45.001 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-24T23:51:45.007 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout enabled 2024-04-24T23:51:45.014 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout active 2024-04-24T23:51:45.014 INFO:teuthology.orchestra.run.smithi110.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:51:45.014 INFO:teuthology.orchestra.run.smithi110.stdout:Host looks OK 2024-04-24T23:51:45.014 INFO:teuthology.orchestra.run.smithi110.stdout:Cluster fsid: 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:45.014 INFO:teuthology.orchestra.run.smithi110.stdout:Acquiring lock 139651739997088 on /run/cephadm/787fa75a-0295-11ef-bc93-c7b262605968.lock 2024-04-24T23:51:45.014 INFO:teuthology.orchestra.run.smithi110.stdout:Lock 139651739997088 acquired on /run/cephadm/787fa75a-0295-11ef-bc93-c7b262605968.lock 2024-04-24T23:51:45.014 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying IP 172.21.15.110 port 3300 ... 2024-04-24T23:51:45.014 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying IP 172.21.15.110 port 6789 ... 2024-04-24T23:51:45.015 INFO:teuthology.orchestra.run.smithi110.stdout:Base mon IP(s) is [172.21.15.110:3300, 172.21.15.110:6789], mon addrv is [v2:172.21.15.110:3300,v1:172.21.15.110:6789] 2024-04-24T23:51:45.017 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.110 metric 100 2024-04-24T23:51:45.017 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.110 metric 100 2024-04-24T23:51:45.020 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T23:51:45.020 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-24T23:51:45.020 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-24T23:51:45.022 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T23:51:45.022 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T23:51:45.023 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:51:45.023 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T23:51:45.023 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout inet6 fe80::ff83:2cf2:308:5b44/64 scope link noprefixroute 2024-04-24T23:51:45.023 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:51:45.023 INFO:teuthology.orchestra.run.smithi110.stdout:Mon IP `172.21.15.110` is in CIDR network `172.21.0.0/20` 2024-04-24T23:51:45.023 INFO:teuthology.orchestra.run.smithi110.stdout:Mon IP `172.21.15.110` is in CIDR network `172.21.0.0/20` 2024-04-24T23:51:45.023 INFO:teuthology.orchestra.run.smithi110.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T23:51:45.024 INFO:teuthology.orchestra.run.smithi110.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T23:51:45.024 INFO:teuthology.orchestra.run.smithi110.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:51:45.465 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:51:45.465 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:51:45.465 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T23:51:45.466 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T23:51:45.466 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-24T23:51:45.466 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:51:45.466 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T23:51:46.345 INFO:teuthology.orchestra.run.smithi110.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:51:46.345 INFO:teuthology.orchestra.run.smithi110.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:51:46.345 INFO:teuthology.orchestra.run.smithi110.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T23:51:47.342 INFO:teuthology.orchestra.run.smithi110.stdout:stat: stdout 167 167 2024-04-24T23:51:47.343 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial keys... 2024-04-24T23:51:48.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQATmylmu4SFKBAAeHvN2/mxCj0Acqv6UBkY2w== 2024-04-24T23:51:49.261 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQAUmylmtkGkKxAAB+J2yFxX99hw66Wx45gHcw== 2024-04-24T23:51:50.126 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQAVmylmLeEoJBAAlhbhR/5ElkR2Y7dQL+D3kw== 2024-04-24T23:51:50.127 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial monmap... 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool for a [v2:172.21.15.110:3300,v1:172.21.15.110:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:setting min_mon_release = quincy 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: set fsid to 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:51:51.069 INFO:teuthology.orchestra.run.smithi110.stdout:Creating mon... 2024-04-24T23:51:51.988 INFO:teuthology.orchestra.run.smithi110.stdout:create mon.a on 2024-04-24T23:51:52.185 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T23:51:52.332 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T23:51:52.520 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-787fa75a-0295-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-787fa75a-0295-11ef-bc93-c7b262605968.target. 2024-04-24T23:51:52.521 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-787fa75a-0295-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-787fa75a-0295-11ef-bc93-c7b262605968.target. 2024-04-24T23:51:52.756 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:51:52.756 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to reset failed state of unit ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T23:51:52.939 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-787fa75a-0295-11ef-bc93-c7b262605968.target.wants/ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-787fa75a-0295-11ef-bc93-c7b262605968@.service. 2024-04-24T23:51:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:53 smithi110 podman[28797]: 2024-04-24 23:51:53.094457723 +0000 UTC m=+0.018451767 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:51:53.471 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:51:53.472 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-04-24T23:51:53.478 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:51:53.479 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-24T23:51:53.479 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-04-24T23:51:53.479 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:51:53.479 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mon to start... 2024-04-24T23:51:53.479 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mon... 2024-04-24T23:51:53.556 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:53 smithi110 podman[28797]: 2024-04-24 23:51:53.375996625 +0000 UTC m=+0.299990645 container init a9cfa1f4ae2801d842edbbcc6d655bb6c5ff335a3e4ed335e3b0fb8fdfd46b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T23:51:53.556 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:53 smithi110 podman[28797]: 2024-04-24 23:51:53.380145109 +0000 UTC m=+0.304139130 container start a9cfa1f4ae2801d842edbbcc6d655bb6c5ff335a3e4ed335e3b0fb8fdfd46b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9) 2024-04-24T23:51:53.556 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:53 smithi110 ceph-mon[28817]: mkfs 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:53.556 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:53 smithi110 ceph-mon[28817]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:51:53.556 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:53 smithi110 bash[28797]: a9cfa1f4ae2801d842edbbcc6d655bb6c5ff335a3e4ed335e3b0fb8fdfd46b24 2024-04-24T23:51:53.556 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:53 smithi110 systemd[1]: Started Ceph mon.a for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:51:54.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout id: 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout services: 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.591391s) 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout data: 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T23:51:54.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.942+0000 7efc28e5b700 1 Processor -- start 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.942+0000 7efc28e5b700 1 -- start start 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.943+0000 7efc28e5b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc24107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.943+0000 7efc28e5b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc241084a0 con 0x7efc24107ab0 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.943+0000 7efc2259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc24107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.943+0000 7efc2259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc24107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40248/0 (socket says 172.21.15.110:40248) 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.943+0000 7efc2259c700 1 -- 172.21.15.110:0/172271494 learned_addr learned my addr 172.21.15.110:0/172271494 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.944+0000 7efc2259c700 1 -- 172.21.15.110:0/172271494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc24108c60 con 0x7efc24107ab0 2024-04-24T23:51:54.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.944+0000 7efc2259c700 1 --2- 172.21.15.110:0/172271494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc24107ed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efc180098a0 tx=0x7efc18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9218a71dd8a70a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.944+0000 7efc2159a700 1 -- 172.21.15.110:0/172271494 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc18004160 con 0x7efc24107ab0 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.944+0000 7efc2159a700 1 -- 172.21.15.110:0/172271494 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc18004300 con 0x7efc24107ab0 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.944+0000 7efc2159a700 1 -- 172.21.15.110:0/172271494 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc180036a0 con 0x7efc24107ab0 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 -- 172.21.15.110:0/172271494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 msgr2=0x7efc24107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 --2- 172.21.15.110:0/172271494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc24107ed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efc180098a0 tx=0x7efc18009bb0 comp rx=0 tx=0).stop 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 -- 172.21.15.110:0/172271494 shutdown_connections 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 --2- 172.21.15.110:0/172271494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc24107ed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 -- 172.21.15.110:0/172271494 >> 172.21.15.110:0/172271494 conn(0x7efc24103090 msgr2=0x7efc241054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 -- 172.21.15.110:0/172271494 shutdown_connections 2024-04-24T23:51:54.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 -- 172.21.15.110:0/172271494 wait complete. 2024-04-24T23:51:54.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 Processor -- start 2024-04-24T23:51:54.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.945+0000 7efc28e5b700 1 -- start start 2024-04-24T23:51:54.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc28e5b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc2407db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:51:54.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc28e5b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc24109190 con 0x7efc24107ab0 2024-04-24T23:51:54.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc2259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc2407db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:51:54.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc2259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc2407db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40256/0 (socket says 172.21.15.110:40256) 2024-04-24T23:51:54.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc2259c700 1 -- 172.21.15.110:0/2466045566 learned_addr learned my addr 172.21.15.110:0/2466045566 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:51:54.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc2259c700 1 -- 172.21.15.110:0/2466045566 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2407e050 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc2259c700 1 --2- 172.21.15.110:0/2466045566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc2407db10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efc18000c00 tx=0x7efc18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc137fe700 1 -- 172.21.15.110:0/2466045566 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc18003db0 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc137fe700 1 -- 172.21.15.110:0/2466045566 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc18011630 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2407c240 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc137fe700 1 -- 172.21.15.110:0/2466045566 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc18011950 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.946+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2407c780 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.947+0000 7efc137fe700 1 -- 172.21.15.110:0/2466045566 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efc1801a460 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.947+0000 7efc137fe700 1 -- 172.21.15.110:0/2466045566 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efc1801b780 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.947+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc24063890 con 0x7efc24107ab0 2024-04-24T23:51:54.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:53.950+0000 7efc137fe700 1 -- 172.21.15.110:0/2466045566 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7efc18031020 con 0x7efc24107ab0 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.019+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7efc24194a70 con 0x7efc24107ab0 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.020+0000 7efc137fe700 1 -- 172.21.15.110:0/2466045566 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7efc1801b9d0 con 0x7efc24107ab0 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.020+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 msgr2=0x7efc2407db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.020+0000 7efc28e5b700 1 --2- 172.21.15.110:0/2466045566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc2407db10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efc18000c00 tx=0x7efc18004750 comp rx=0 tx=0).stop 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.020+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 shutdown_connections 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.020+0000 7efc28e5b700 1 --2- 172.21.15.110:0/2466045566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc24107ab0 0x7efc2407db10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.020+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 >> 172.21.15.110:0/2466045566 conn(0x7efc24103090 msgr2=0x7efc24191db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.020+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 shutdown_connections 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.020+0000 7efc28e5b700 1 -- 172.21.15.110:0/2466045566 wait complete. 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:mon is available 2024-04-24T23:51:54.584 INFO:teuthology.orchestra.run.smithi110.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: monmap epoch 1 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: last_changed 2024-04-24T23:51:50.484996+0000 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: created 2024-04-24T23:51:50.484996+0000 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: min_mon_release 19 (squid) 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: election_strategy: 1 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-04-24T23:51:54.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: fsmap 2024-04-24T23:51:54.686 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:51:54.686 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: mgrmap e1: no daemons active 2024-04-24T23:51:54.686 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:54 smithi110 ceph-mon[28817]: from='client.? 172.21.15.110:0/2466045566' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T23:51:55.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:51:55.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:51:55.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout fsid = 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:55.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.110:3300,v1:172.21.15.110:6789] 2024-04-24T23:51:55.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:51:55.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.966+0000 7f066df5a700 1 Processor -- start 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.966+0000 7f066df5a700 1 -- start start 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.966+0000 7f066df5a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f0668106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.966+0000 7f066df5a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0668107330 con 0x7f0668106940 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.967+0000 7f06677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f0668106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.967+0000 7f06677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f0668106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40270/0 (socket says 172.21.15.110:40270) 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.967+0000 7f06677fe700 1 -- 172.21.15.110:0/4218170251 learned_addr learned my addr 172.21.15.110:0/4218170251 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.967+0000 7f06677fe700 1 -- 172.21.15.110:0/4218170251 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668107b40 con 0x7f0668106940 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.967+0000 7f06677fe700 1 --2- 172.21.15.110:0/4218170251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f0668106d60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f06580098a0 tx=0x7f0658009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a391dfc6edc95b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.967+0000 7f0666ffd700 1 -- 172.21.15.110:0/4218170251 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658004160 con 0x7f0668106940 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.967+0000 7f0666ffd700 1 -- 172.21.15.110:0/4218170251 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0658004300 con 0x7f0668106940 2024-04-24T23:51:55.655 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.967+0000 7f0666ffd700 1 -- 172.21.15.110:0/4218170251 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06580036a0 con 0x7f0668106940 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 -- 172.21.15.110:0/4218170251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 msgr2=0x7f0668106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 --2- 172.21.15.110:0/4218170251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f0668106d60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f06580098a0 tx=0x7f0658009bb0 comp rx=0 tx=0).stop 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 -- 172.21.15.110:0/4218170251 shutdown_connections 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 --2- 172.21.15.110:0/4218170251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f0668106d60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 -- 172.21.15.110:0/4218170251 >> 172.21.15.110:0/4218170251 conn(0x7f0668101ea0 msgr2=0x7f0668104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 -- 172.21.15.110:0/4218170251 shutdown_connections 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 -- 172.21.15.110:0/4218170251 wait complete. 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 Processor -- start 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.968+0000 7f066df5a700 1 -- start start 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f066df5a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f06681907c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f066df5a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0668108070 con 0x7f0668106940 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f06677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f06681907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f06677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f06681907c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40278/0 (socket says 172.21.15.110:40278) 2024-04-24T23:51:55.656 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f06677fe700 1 -- 172.21.15.110:0/3201198105 learned_addr learned my addr 172.21.15.110:0/3201198105 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f06677fe700 1 -- 172.21.15.110:0/3201198105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668190d00 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f06677fe700 1 --2- 172.21.15.110:0/3201198105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f06681907c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f06657fa700 1 -- 172.21.15.110:0/3201198105 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658003db0 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f06657fa700 1 -- 172.21.15.110:0/3201198105 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0658011630 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f06657fa700 1 -- 172.21.15.110:0/3201198105 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658011950 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066818eef0 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.969+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066818f430 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.970+0000 7f06657fa700 1 -- 172.21.15.110:0/3201198105 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0658011af0 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.970+0000 7f06657fa700 1 -- 172.21.15.110:0/3201198105 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f065801c450 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.970+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0668063890 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:54.973+0000 7f06657fa700 1 -- 172.21.15.110:0/3201198105 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0658032020 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.039+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f066818f6b0 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.043+0000 7f06657fa700 1 -- 172.21.15.110:0/3201198105 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f0658004260 con 0x7f0668106940 2024-04-24T23:51:55.657 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.043+0000 7f06657fa700 1 -- 172.21.15.110:0/3201198105 <== mon.0 v2:172.21.15.110:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f065801c6a0 con 0x7f0668106940 2024-04-24T23:51:55.658 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.044+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 msgr2=0x7f06681907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:51:55.658 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.044+0000 7f066df5a700 1 --2- 172.21.15.110:0/3201198105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f06681907c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658004750 comp rx=0 tx=0).stop 2024-04-24T23:51:55.658 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.044+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 shutdown_connections 2024-04-24T23:51:55.658 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.044+0000 7f066df5a700 1 --2- 172.21.15.110:0/3201198105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0668106940 0x7f06681907c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:51:55.658 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.044+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 >> 172.21.15.110:0/3201198105 conn(0x7f0668101ea0 msgr2=0x7f066818e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:51:55.658 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.044+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 shutdown_connections 2024-04-24T23:51:55.658 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:55.044+0000 7f066df5a700 1 -- 172.21.15.110:0/3201198105 wait complete. 2024-04-24T23:51:55.658 INFO:teuthology.orchestra.run.smithi110.stdout:Generating new minimal ceph.conf... 2024-04-24T23:51:55.761 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:55 smithi110 ceph-mon[28817]: from='client.? 172.21.15.110:0/3201198105' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:51:55.762 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:55 smithi110 ceph-mon[28817]: from='client.? 172.21.15.110:0/3201198105' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T23:51:56.711 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:56 smithi110 ceph-mon[28817]: from='client.? 172.21.15.110:0/2846542514' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:51:56.714 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.070+0000 7ff53385d700 1 Processor -- start 2024-04-24T23:51:56.720 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.070+0000 7ff53385d700 1 -- start start 2024-04-24T23:51:56.720 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff53385d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c10a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:51:56.720 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff53385d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c07be10 con 0x7ff52c1084e0 2024-04-24T23:51:56.720 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff5315f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c10a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:51:56.720 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff5315f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c10a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40292/0 (socket says 172.21.15.110:40292) 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff5315f9700 1 -- 172.21.15.110:0/1185764737 learned_addr learned my addr 172.21.15.110:0/1185764737 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff5315f9700 1 -- 172.21.15.110:0/1185764737 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c10ae10 con 0x7ff52c1084e0 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff5315f9700 1 --2- 172.21.15.110:0/1185764737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c10a8d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff524009e80 tx=0x7ff52400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec57a97d2d49c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff523fff700 1 -- 172.21.15.110:0/1185764737 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff524017070 con 0x7ff52c1084e0 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff523fff700 1 -- 172.21.15.110:0/1185764737 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff52400b8c0 con 0x7ff52c1084e0 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.071+0000 7ff523fff700 1 -- 172.21.15.110:0/1185764737 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff524004520 con 0x7ff52c1084e0 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.072+0000 7ff53385d700 1 -- 172.21.15.110:0/1185764737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 msgr2=0x7ff52c10a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.072+0000 7ff53385d700 1 --2- 172.21.15.110:0/1185764737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c10a8d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff524009e80 tx=0x7ff52400b330 comp rx=0 tx=0).stop 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.072+0000 7ff53385d700 1 -- 172.21.15.110:0/1185764737 shutdown_connections 2024-04-24T23:51:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.072+0000 7ff53385d700 1 --2- 172.21.15.110:0/1185764737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c10a8d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.072+0000 7ff53385d700 1 -- 172.21.15.110:0/1185764737 >> 172.21.15.110:0/1185764737 conn(0x7ff52c101ed0 msgr2=0x7ff52c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.072+0000 7ff53385d700 1 -- 172.21.15.110:0/1185764737 shutdown_connections 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.072+0000 7ff53385d700 1 -- 172.21.15.110:0/1185764737 wait complete. 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff53385d700 1 Processor -- start 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff53385d700 1 -- start start 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff53385d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c107d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff53385d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c10bdd0 con 0x7ff52c1084e0 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff5315f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff5315f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c107d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40308/0 (socket says 172.21.15.110:40308) 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff5315f9700 1 -- 172.21.15.110:0/2846542514 learned_addr learned my addr 172.21.15.110:0/2846542514 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:51:56.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff5315f9700 1 -- 172.21.15.110:0/2846542514 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c1082a0 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff5315f9700 1 --2- 172.21.15.110:0/2846542514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c107d60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b7b0 tx=0x7ff524004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff5227fc700 1 -- 172.21.15.110:0/2846542514 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff524020020 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c106490 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.073+0000 7ff5227fc700 1 -- 172.21.15.110:0/2846542514 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff52400bba0 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.074+0000 7ff5227fc700 1 -- 172.21.15.110:0/2846542514 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff524017040 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.074+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c1069d0 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.074+0000 7ff5227fc700 1 -- 172.21.15.110:0/2846542514 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff524023020 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.074+0000 7ff5227fc700 1 -- 172.21.15.110:0/2846542514 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff52400f050 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.075+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52c063890 con 0x7ff52c1084e0 2024-04-24T23:51:56.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.077+0000 7ff5227fc700 1 -- 172.21.15.110:0/2846542514 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff52402a020 con 0x7ff52c1084e0 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.144+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff52c106d10 con 0x7ff52c1084e0 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.144+0000 7ff5227fc700 1 -- 172.21.15.110:0/2846542514 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7ff52400b860 con 0x7ff52c1084e0 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.145+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 msgr2=0x7ff52c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.145+0000 7ff53385d700 1 --2- 172.21.15.110:0/2846542514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c107d60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b7b0 tx=0x7ff524004670 comp rx=0 tx=0).stop 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.145+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 shutdown_connections 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.145+0000 7ff53385d700 1 --2- 172.21.15.110:0/2846542514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff52c1084e0 0x7ff52c107d60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.145+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 >> 172.21.15.110:0/2846542514 conn(0x7ff52c101ed0 msgr2=0x7ff52c1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.145+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 shutdown_connections 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:56.145+0000 7ff53385d700 1 -- 172.21.15.110:0/2846542514 wait complete. 2024-04-24T23:51:56.724 INFO:teuthology.orchestra.run.smithi110.stdout:Restarting the monitor... 2024-04-24T23:51:56.983 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:56 smithi110 systemd[1]: Stopping Ceph mon.a for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:51:57.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:56 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a[28813]: 2024-04-24T23:51:56.982+0000 7fda8a578700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:51:57.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:56 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a[28813]: 2024-04-24T23:51:56.982+0000 7fda8a578700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T23:51:57.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:57 smithi110 podman[29061]: 2024-04-24 23:51:57.207684277 +0000 UTC m=+0.328759782 container died a9cfa1f4ae2801d842edbbcc6d655bb6c5ff335a3e4ed335e3b0fb8fdfd46b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-24T23:51:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:57 smithi110 podman[29061]: 2024-04-24 23:51:57.557354995 +0000 UTC m=+0.678430507 container cleanup a9cfa1f4ae2801d842edbbcc6d655bb6c5ff335a3e4ed335e3b0fb8fdfd46b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:51:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:57 smithi110 bash[29061]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:51:58.183 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:57 smithi110 podman[29075]: 2024-04-24 23:51:57.825219862 +0000 UTC m=+0.615993734 container remove a9cfa1f4ae2801d842edbbcc6d655bb6c5ff335a3e4ed335e3b0fb8fdfd46b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T23:51:58.183 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:57 smithi110 systemd[1]: ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T23:51:58.183 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:57 smithi110 systemd[1]: Stopped Ceph mon.a for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:51:58.183 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:57 smithi110 systemd[1]: Starting Ceph mon.a for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:51:58.443 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 podman[29168]: 2024-04-24 23:51:58.184281193 +0000 UTC m=+0.103660913 container create 0ca456db64e894722112587243b9548b4f4369f2fd84b188dffa298ae2b74521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:51:58.443 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 podman[29168]: 2024-04-24 23:51:58.099078041 +0000 UTC m=+0.018457774 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:51:58.443 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 podman[29168]: 2024-04-24 23:51:58.353224705 +0000 UTC m=+0.272604427 container init 0ca456db64e894722112587243b9548b4f4369f2fd84b188dffa298ae2b74521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:51:58.443 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 podman[29168]: 2024-04-24 23:51:58.356422657 +0000 UTC m=+0.275802381 container start 0ca456db64e894722112587243b9548b4f4369f2fd84b188dffa298ae2b74521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:51:58.443 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: pidfile_write: ignore empty --pid-file 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: load: jerasure load: lrc 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Git sha 0 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: DB SUMMARY 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: DB Session ID: EDEJFF0FYA8BLAAE787V 2024-04-24T23:51:58.444 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103790 ; 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:51:58.445 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.env: 0x55e029037c60 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.info_log: 0x55e02ada9aa0 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.statistics: (nil) 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.use_fsync: 0 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:51:58.446 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.db_log_dir: 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.wal_dir: 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:51:58.447 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:51:58.448 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:51:58.448 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.write_buffer_manager: 0x55e02ad7c1e0 2024-04-24T23:51:58.448 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:51:58.448 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:51:58.448 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:51:58.448 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:51:58.448 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:51:58.448 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.unordered_write: 0 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.row_cache: None 2024-04-24T23:51:58.451 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.wal_filter: None 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.wal_compression: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:51:58.452 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:51:58.453 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_open_files: -1 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:51:58.454 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Compression algorithms supported: 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: kZSTD supported: 0 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T23:51:58.455 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.merge_operator: 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_filter: None 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e02ada9ba0) 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:51:58.456 INFO:journalctl@ceph.mon.a.smithi110.stdout: index_type: 0 2024-04-24T23:51:58.457 INFO:journalctl@ceph.mon.a.smithi110.stdout: data_block_index_type: 0 2024-04-24T23:51:58.457 INFO:journalctl@ceph.mon.a.smithi110.stdout: index_shortening: 1 2024-04-24T23:51:58.457 INFO:journalctl@ceph.mon.a.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:51:58.457 INFO:journalctl@ceph.mon.a.smithi110.stdout: checksum: 4 2024-04-24T23:51:58.457 INFO:journalctl@ceph.mon.a.smithi110.stdout: no_block_cache: 0 2024-04-24T23:51:58.457 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_cache: 0x55e02adad090 2024-04-24T23:51:58.457 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:51:58.457 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_cache_options: 2024-04-24T23:51:58.458 INFO:journalctl@ceph.mon.a.smithi110.stdout: capacity : 536870912 2024-04-24T23:51:58.458 INFO:journalctl@ceph.mon.a.smithi110.stdout: num_shard_bits : 4 2024-04-24T23:51:58.458 INFO:journalctl@ceph.mon.a.smithi110.stdout: strict_capacity_limit : 0 2024-04-24T23:51:58.458 INFO:journalctl@ceph.mon.a.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:51:58.458 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_cache_compressed: (nil) 2024-04-24T23:51:58.459 INFO:journalctl@ceph.mon.a.smithi110.stdout: persistent_cache: (nil) 2024-04-24T23:51:58.459 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_size: 4096 2024-04-24T23:51:58.459 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_size_deviation: 10 2024-04-24T23:51:58.459 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_restart_interval: 16 2024-04-24T23:51:58.459 INFO:journalctl@ceph.mon.a.smithi110.stdout: index_block_restart_interval: 1 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: metadata_block_size: 4096 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: partition_filters: 0 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: use_delta_encoding: 1 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: filter_policy: bloomfilter 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: whole_key_filtering: 1 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: verify_compression: 0 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: format_version: 5 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: enable_index_compression: 1 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: block_align: 0 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: max_auto_readahead_size: 262144 2024-04-24T23:51:58.460 INFO:journalctl@ceph.mon.a.smithi110.stdout: prepopulate_block_cache: 0 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression: NoCompression 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.num_levels: 7 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:51:58.461 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:51:58.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:51:58.463 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:51:58.464 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:51:58.464 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:51:58.464 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:51:58.464 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:51:58.464 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:51:58.464 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:51:58.464 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:51:58.464 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:51:58.465 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:51:58.466 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.ttl: 2592000 2024-04-24T23:51:58.467 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:51:58.468 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:51:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:51:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T23:51:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T23:51:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e4d82432-b7e1-4523-b424-5805d9105393 2024-04-24T23:51:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714002718385021, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T23:51:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T23:51:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714002718385962, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98963, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 96960, "index_size": 285, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13919, "raw_average_key_size": 49, "raw_value_size": 89356, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714002718, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e4d82432-b7e1-4523-b424-5805d9105393", "db_session_id": "EDEJFF0FYA8BLAAE787V", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:51:58.469 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714002718386048, "job": 1, "event": "recovery_finished"} 2024-04-24T23:51:58.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T23:51:58.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:51:58.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e02ae9e000 2024-04-24T23:51:58.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: DB pointer 0x55e02ae88000 2024-04-24T23:51:58.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:51:58.470 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** DB Stats ** 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-04-24T23:51:58.471 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** Compaction Stats [default] ** 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: L0 2/0 98.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: Sum 2/0 98.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** Compaction Stats [default] ** 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 160.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:51:58.472 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: Cumulative compaction: 0.00 GB write, 32.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:51:58.473 INFO:journalctl@ceph.mon.a.smithi110.stdout: Interval compaction: 0.00 GB write, 32.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout: Block cache BinnedLRUCache@0x55e02adad090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: starting mon.a rank 0 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a@-1(???) e1 preinit fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a@-1(???).mds e1 new map 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a@-1(???).mds e1 print_map 2024-04-24T23:51:58.474 INFO:journalctl@ceph.mon.a.smithi110.stdout: e1 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout: legacy client fscid: -1 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout: 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout: No filesystems configured 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:51:58.475 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: monmap epoch 1 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: last_changed 2024-04-24T23:51:50.484996+0000 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: created 2024-04-24T23:51:50.484996+0000 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: min_mon_release 19 (squid) 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: election_strategy: 1 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: fsmap 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:51:58.476 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 ceph-mon[29187]: mgrmap e1: no daemons active 2024-04-24T23:51:58.477 INFO:teuthology.orchestra.run.smithi110.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T23:51:58.694 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 bash[29168]: 0ca456db64e894722112587243b9548b4f4369f2fd84b188dffa298ae2b74521 2024-04-24T23:51:58.694 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:58 smithi110 systemd[1]: Started Ceph mon.a for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:51:59.499 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.881+0000 7f216df7e700 1 Processor -- start 2024-04-24T23:51:59.499 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.882+0000 7f216df7e700 1 -- start start 2024-04-24T23:51:59.499 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.882+0000 7f216df7e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f2168107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:51:59.499 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.882+0000 7f216df7e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21681084a0 con 0x7f2168107ab0 2024-04-24T23:51:59.499 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.882+0000 7f21677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f2168107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:51:59.499 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.882+0000 7f21677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f2168107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50904/0 (socket says 172.21.15.110:50904) 2024-04-24T23:51:59.499 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.882+0000 7f21677fe700 1 -- 172.21.15.110:0/1078584228 learned_addr learned my addr 172.21.15.110:0/1078584228 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:51:59.499 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.882+0000 7f21677fe700 1 -- 172.21.15.110:0/1078584228 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2168108c60 con 0x7f2168107ab0 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.882+0000 7f21677fe700 1 --2- 172.21.15.110:0/1078584228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f2168107ed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f21500098a0 tx=0x7f2150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa4b1162c3c85c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f21667fc700 1 -- 172.21.15.110:0/1078584228 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2150004160 con 0x7f2168107ab0 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f21667fc700 1 -- 172.21.15.110:0/1078584228 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2150004300 con 0x7f2168107ab0 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f21667fc700 1 -- 172.21.15.110:0/1078584228 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2150011a80 con 0x7f2168107ab0 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f216df7e700 1 -- 172.21.15.110:0/1078584228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 msgr2=0x7f2168107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f216df7e700 1 --2- 172.21.15.110:0/1078584228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f2168107ed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f21500098a0 tx=0x7f2150009bb0 comp rx=0 tx=0).stop 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f216df7e700 1 -- 172.21.15.110:0/1078584228 shutdown_connections 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f216df7e700 1 --2- 172.21.15.110:0/1078584228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f2168107ed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f216df7e700 1 -- 172.21.15.110:0/1078584228 >> 172.21.15.110:0/1078584228 conn(0x7f2168103090 msgr2=0x7f21681054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f216df7e700 1 -- 172.21.15.110:0/1078584228 shutdown_connections 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.883+0000 7f216df7e700 1 -- 172.21.15.110:0/1078584228 wait complete. 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.884+0000 7f216df7e700 1 Processor -- start 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.884+0000 7f216df7e700 1 -- start start 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.884+0000 7f216df7e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f216819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:51:59.500 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.884+0000 7f216df7e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f216810c010 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.884+0000 7f21677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f216819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.884+0000 7f21677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f216819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50918/0 (socket says 172.21.15.110:50918) 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.884+0000 7f21677fe700 1 -- 172.21.15.110:0/1291089798 learned_addr learned my addr 172.21.15.110:0/1291089798 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.884+0000 7f21677fe700 1 -- 172.21.15.110:0/1291089798 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216819bf60 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.885+0000 7f21677fe700 1 --2- 172.21.15.110:0/1291089798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f216819ba20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2150006af0 tx=0x7f2150011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.885+0000 7f2164ff9700 1 -- 172.21.15.110:0/1291089798 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2150003930 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.885+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216819c1f0 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.885+0000 7f2164ff9700 1 -- 172.21.15.110:0/1291089798 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2150003ad0 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.885+0000 7f2164ff9700 1 -- 172.21.15.110:0/1291089798 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21500195e0 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.885+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216819eee0 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.885+0000 7f2164ff9700 1 -- 172.21.15.110:0/1291089798 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2150019c90 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.885+0000 7f2164ff9700 1 -- 172.21.15.110:0/1291089798 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2150024730 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.886+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2168063890 con 0x7f2168107ab0 2024-04-24T23:51:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.888+0000 7f2164ff9700 1 -- 172.21.15.110:0/1291089798 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2150043dd0 con 0x7f2168107ab0 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.956+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f216819f4a0 con 0x7f2168107ab0 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.957+0000 7f2164ff9700 1 -- 172.21.15.110:0/1291089798 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f2150032020 con 0x7f2168107ab0 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.958+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 msgr2=0x7f216819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.958+0000 7f216df7e700 1 --2- 172.21.15.110:0/1291089798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f216819ba20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2150006af0 tx=0x7f2150011b70 comp rx=0 tx=0).stop 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.958+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 shutdown_connections 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.958+0000 7f216df7e700 1 --2- 172.21.15.110:0/1291089798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2168107ab0 0x7f216819ba20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.958+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 >> 172.21.15.110:0/1291089798 conn(0x7f2168103090 msgr2=0x7f2168103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.958+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 shutdown_connections 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:51:58.958+0000 7f216df7e700 1 -- 172.21.15.110:0/1291089798 wait complete. 2024-04-24T23:51:59.502 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T23:51:59.517 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:51:59.517 INFO:teuthology.orchestra.run.smithi110.stdout:Creating mgr... 2024-04-24T23:51:59.517 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T23:51:59.518 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T23:51:59.757 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:51:59.757 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to reset failed state of unit ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T23:51:59.905 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-787fa75a-0295-11ef-bc93-c7b262605968.target.wants/ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-787fa75a-0295-11ef-bc93-c7b262605968@.service. 2024-04-24T23:52:00.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:51:59 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1291089798' entity='client.admin' 2024-04-24T23:52:00.439 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:00 smithi110 podman[29404]: 2024-04-24 23:52:00.358673463 +0000 UTC m=+0.330444130 container start 6592987f0c90f8aae6e8f29bfa5ba2705cc35e0d455928314b48ad2bf3348121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:52:00.439 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:00 smithi110 bash[29404]: 6592987f0c90f8aae6e8f29bfa5ba2705cc35e0d455928314b48ad2bf3348121 2024-04-24T23:52:00.453 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:52:00.453 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-04-24T23:52:00.460 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:52:00.460 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-24T23:52:00.460 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-04-24T23:52:00.460 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:52:00.468 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:52:00.468 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-04-24T23:52:00.475 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:52:00.476 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-24T23:52:00.476 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-04-24T23:52:00.476 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T23:52:00.476 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr to start... 2024-04-24T23:52:00.476 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr... 2024-04-24T23:52:00.693 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:00 smithi110 systemd[1]: Started Ceph mgr.x for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:52:00.693 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:00.509+0000 7ff6f563e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:52:00.693 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:00.693+0000 7ff6f563e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:52:00.988 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:00.798+0000 7ff6f563e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:52:01.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:01 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4277502161' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:52:01.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:01 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:01.125+0000 7ff6f563e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:52:01.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:01 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:01.209+0000 7ff6f563e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:52:01.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:52:01.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:52:01.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-24T23:52:01.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:52:01.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:52:01.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:52:01.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:52:01.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:01.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:51:53.430606+0000", 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:52:01.580 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.909+0000 7f6c76538700 1 Processor -- start 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.909+0000 7f6c76538700 1 -- start start 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.910+0000 7f6c76538700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.910+0000 7f6c76538700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7007c890 con 0x7f6c7007de10 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.910+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.910+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50960/0 (socket says 172.21.15.110:50960) 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.910+0000 7f6c6ffff700 1 -- 172.21.15.110:0/4133514501 learned_addr learned my addr 172.21.15.110:0/4133514501 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.910+0000 7f6c6ffff700 1 -- 172.21.15.110:0/4133514501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7007ca10 con 0x7f6c7007de10 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.910+0000 7f6c6ffff700 1 --2- 172.21.15.110:0/4133514501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7007c2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6c600098a0 tx=0x7f6c60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192dec16c7165f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.910+0000 7f6c6effd700 1 -- 172.21.15.110:0/4133514501 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c60004160 con 0x7f6c7007de10 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c6effd700 1 -- 172.21.15.110:0/4133514501 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6c60004300 con 0x7f6c7007de10 2024-04-24T23:52:01.581 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c6effd700 1 -- 172.21.15.110:0/4133514501 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c60011a80 con 0x7f6c7007de10 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c76538700 1 -- 172.21.15.110:0/4133514501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 msgr2=0x7f6c7007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c76538700 1 --2- 172.21.15.110:0/4133514501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7007c2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6c600098a0 tx=0x7f6c60009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c76538700 1 -- 172.21.15.110:0/4133514501 shutdown_connections 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c76538700 1 --2- 172.21.15.110:0/4133514501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7007c2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c76538700 1 -- 172.21.15.110:0/4133514501 >> 172.21.15.110:0/4133514501 conn(0x7f6c70103090 msgr2=0x7f6c701054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c76538700 1 -- 172.21.15.110:0/4133514501 shutdown_connections 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.911+0000 7f6c76538700 1 -- 172.21.15.110:0/4133514501 wait complete. 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.912+0000 7f6c76538700 1 Processor -- start 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.912+0000 7f6c76538700 1 -- start start 2024-04-24T23:52:01.582 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.912+0000 7f6c76538700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:01.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.912+0000 7f6c76538700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7010df80 con 0x7f6c7007de10 2024-04-24T23:52:01.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:01.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7019db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50966/0 (socket says 172.21.15.110:50966) 2024-04-24T23:52:01.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6ffff700 1 -- 172.21.15.110:0/4277502161 learned_addr learned my addr 172.21.15.110:0/4277502161 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:01.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6ffff700 1 -- 172.21.15.110:0/4277502161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7019e070 con 0x7f6c7007de10 2024-04-24T23:52:01.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6ffff700 1 --2- 172.21.15.110:0/4277502161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7019db30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6c60006af0 tx=0x7f6c60011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:01.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6d7fa700 1 -- 172.21.15.110:0/4277502161 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c600039e0 con 0x7f6c7007de10 2024-04-24T23:52:01.583 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6d7fa700 1 -- 172.21.15.110:0/4277502161 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6c60003b80 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7019e300 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6d7fa700 1 -- 172.21.15.110:0/4277502161 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c600196a0 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c701a0ff0 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6d7fa700 1 -- 172.21.15.110:0/4277502161 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6c6002d440 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.913+0000 7f6c6d7fa700 1 -- 172.21.15.110:0/4277502161 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6c60024800 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.914+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c70063890 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.916+0000 7f6c6d7fa700 1 -- 172.21.15.110:0/4277502161 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6c6001ab50 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.988+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6c701a11c0 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.989+0000 7f6c6d7fa700 1 -- 172.21.15.110:0/4277502161 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6c60024a00 con 0x7f6c7007de10 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.990+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 msgr2=0x7f6c7019db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:01.584 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.990+0000 7f6c76538700 1 --2- 172.21.15.110:0/4277502161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7019db30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6c60006af0 tx=0x7f6c60011c40 comp rx=0 tx=0).stop 2024-04-24T23:52:01.585 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.990+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 shutdown_connections 2024-04-24T23:52:01.585 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.990+0000 7f6c76538700 1 --2- 172.21.15.110:0/4277502161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c7007de10 0x7f6c7019db30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:01.585 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.990+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 >> 172.21.15.110:0/4277502161 conn(0x7f6c70103090 msgr2=0x7f6c70103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:01.585 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.990+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 shutdown_connections 2024-04-24T23:52:01.585 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:00.990+0000 7f6c76538700 1 -- 172.21.15.110:0/4277502161 wait complete. 2024-04-24T23:52:01.585 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (1/15)... 2024-04-24T23:52:02.736 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:02.398+0000 7ff6f563e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:52:02.736 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:02.568+0000 7ff6f563e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:52:02.736 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:02.736+0000 7ff6f563e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:52:03.285 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:03.023+0000 7ff6f563e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:52:03.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:03.114+0000 7ff6f563e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:52:03.617 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:03.312+0000 7ff6f563e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:52:03.617 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:03.500+0000 7ff6f563e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:52:04.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:04 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1887579145' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:52:04.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:04.131+0000 7ff6f563e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:52:04.652 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:52:04.652 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:52:04.652 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-24T23:52:04.652 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:52:04.652 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:04.653 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:52:04.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:52:04.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:04.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:52:04.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:04.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:52:04.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:52:04.654 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:52:04.663 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:52:04.664 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:51:53.430606+0000", 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:52:04.665 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.008+0000 7f4d7cee5700 1 Processor -- start 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.008+0000 7f4d7cee5700 1 -- start start 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.008+0000 7f4d7cee5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d78107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.008+0000 7f4d7cee5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78108480 con 0x7f4d78107a90 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.009+0000 7f4d7659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d78107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.009+0000 7f4d7659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d78107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50980/0 (socket says 172.21.15.110:50980) 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.009+0000 7f4d7659c700 1 -- 172.21.15.110:0/657996601 learned_addr learned my addr 172.21.15.110:0/657996601 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.009+0000 7f4d7659c700 1 -- 172.21.15.110:0/657996601 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d78108c40 con 0x7f4d78107a90 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.009+0000 7f4d7659c700 1 --2- 172.21.15.110:0/657996601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d78107eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4d680098a0 tx=0x7f4d68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ae42fc84a93b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.009+0000 7f4d7559a700 1 -- 172.21.15.110:0/657996601 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d68004160 con 0x7f4d78107a90 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7559a700 1 -- 172.21.15.110:0/657996601 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4d68004300 con 0x7f4d78107a90 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7559a700 1 -- 172.21.15.110:0/657996601 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d68011a80 con 0x7f4d78107a90 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7cee5700 1 -- 172.21.15.110:0/657996601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 msgr2=0x7f4d78107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7cee5700 1 --2- 172.21.15.110:0/657996601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d78107eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4d680098a0 tx=0x7f4d68009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:04.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7cee5700 1 -- 172.21.15.110:0/657996601 shutdown_connections 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7cee5700 1 --2- 172.21.15.110:0/657996601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d78107eb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7cee5700 1 -- 172.21.15.110:0/657996601 >> 172.21.15.110:0/657996601 conn(0x7f4d78103050 msgr2=0x7f4d78105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7cee5700 1 -- 172.21.15.110:0/657996601 shutdown_connections 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.010+0000 7f4d7cee5700 1 -- 172.21.15.110:0/657996601 wait complete. 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.011+0000 7f4d7cee5700 1 Processor -- start 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.011+0000 7f4d7cee5700 1 -- start start 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.011+0000 7f4d7cee5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d7807db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.011+0000 7f4d7cee5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810bff0 con 0x7f4d78107a90 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d7659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d7807db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d7659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d7807db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50994/0 (socket says 172.21.15.110:50994) 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d7659c700 1 -- 172.21.15.110:0/1887579145 learned_addr learned my addr 172.21.15.110:0/1887579145 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d7659c700 1 -- 172.21.15.110:0/1887579145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7807e050 con 0x7f4d78107a90 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d7659c700 1 --2- 172.21.15.110:0/1887579145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d7807db10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4d68006af0 tx=0x7f4d68011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:04.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d677fe700 1 -- 172.21.15.110:0/1887579145 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d680039e0 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d677fe700 1 -- 172.21.15.110:0/1887579145 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4d68003b80 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7807c240 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d677fe700 1 -- 172.21.15.110:0/1887579145 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d680196a0 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.012+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7807c780 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.013+0000 7f4d677fe700 1 -- 172.21.15.110:0/1887579145 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4d6802d440 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.013+0000 7f4d677fe700 1 -- 172.21.15.110:0/1887579145 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4d680247c0 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.013+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d78063890 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.017+0000 7f4d677fe700 1 -- 172.21.15.110:0/1887579145 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4d68043dd0 con 0x7f4d78107a90 2024-04-24T23:52:04.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.087+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4d7807ca30 con 0x7f4d78107a90 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.087+0000 7f4d677fe700 1 -- 172.21.15.110:0/1887579145 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4d68032020 con 0x7f4d78107a90 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.088+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 msgr2=0x7f4d7807db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.088+0000 7f4d7cee5700 1 --2- 172.21.15.110:0/1887579145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d7807db10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4d68006af0 tx=0x7f4d68011c40 comp rx=0 tx=0).stop 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.088+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 shutdown_connections 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.088+0000 7f4d7cee5700 1 --2- 172.21.15.110:0/1887579145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d78107a90 0x7f4d7807db10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.088+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 >> 172.21.15.110:0/1887579145 conn(0x7f4d78103050 msgr2=0x7f4d78103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.088+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 shutdown_connections 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:04.088+0000 7f4d7cee5700 1 -- 172.21.15.110:0/1887579145 wait complete. 2024-04-24T23:52:04.669 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (2/15)... 2024-04-24T23:52:04.853 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:04.555+0000 7ff6f563e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:52:04.853 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:04.654+0000 7ff6f563e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:52:04.853 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:04.853+0000 7ff6f563e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:52:05.691 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:05.363+0000 7ff6f563e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:52:05.691 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:05.452+0000 7ff6f563e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:52:06.028 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:05.691+0000 7ff6f563e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:52:06.028 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:05.822+0000 7ff6f563e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:52:06.028 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:05.915+0000 7ff6f563e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:52:06.028 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:06.028+0000 7ff6f563e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:52:06.285 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:06.118+0000 7ff6f563e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:52:07.082 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:06.942+0000 7ff6f563e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:52:07.535 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:07.160+0000 7ff6f563e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: Activating manager daemon x 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: mgrmap e2: x(active, starting, since 0.00295987s) 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: Manager daemon x is now available 2024-04-24T23:52:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:52:07.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:52:07.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' 2024-04-24T23:52:07.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' 2024-04-24T23:52:07.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:07 smithi110 ceph-mon[29187]: from='mgr.14100 172.21.15.110:0/3259565412' entity='mgr.x' 2024-04-24T23:52:07.900 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:52:07.900 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:52:07.900 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-24T23:52:07.900 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:52:07.900 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:52:07.900 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:52:07.900 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:52:07.900 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:07.901 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:52:07.902 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:52:07.903 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:51:53.430606+0000", 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd6939a700 1 Processor -- start 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd6939a700 1 -- start start 2024-04-24T23:52:07.904 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd6939a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd64108300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd6939a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641088d0 con 0x7fcd64107f00 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd62ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd64108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd62ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd64108300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43934/0 (socket says 172.21.15.110:43934) 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd62ffd700 1 -- 172.21.15.110:0/619289026 learned_addr learned my addr 172.21.15.110:0/619289026 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd62ffd700 1 -- 172.21.15.110:0/619289026 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd64108a50 con 0x7fcd64107f00 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.182+0000 7fcd62ffd700 1 --2- 172.21.15.110:0/619289026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd64108300 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcd540098a0 tx=0x7fcd54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4035ba2410c8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd61ffb700 1 -- 172.21.15.110:0/619289026 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd54004160 con 0x7fcd64107f00 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd61ffb700 1 -- 172.21.15.110:0/619289026 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcd54004300 con 0x7fcd64107f00 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd61ffb700 1 -- 172.21.15.110:0/619289026 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd54011a80 con 0x7fcd64107f00 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd6939a700 1 -- 172.21.15.110:0/619289026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 msgr2=0x7fcd64108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd6939a700 1 --2- 172.21.15.110:0/619289026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd64108300 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcd540098a0 tx=0x7fcd54009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd6939a700 1 -- 172.21.15.110:0/619289026 shutdown_connections 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd6939a700 1 --2- 172.21.15.110:0/619289026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd64108300 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:07.905 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd6939a700 1 -- 172.21.15.110:0/619289026 >> 172.21.15.110:0/619289026 conn(0x7fcd64101630 msgr2=0x7fcd64103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd6939a700 1 -- 172.21.15.110:0/619289026 shutdown_connections 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.183+0000 7fcd6939a700 1 -- 172.21.15.110:0/619289026 wait complete. 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.184+0000 7fcd6939a700 1 Processor -- start 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.184+0000 7fcd6939a700 1 -- start start 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.184+0000 7fcd6939a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd6419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.184+0000 7fcd6939a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641068d0 con 0x7fcd64107f00 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.184+0000 7fcd62ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd6419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.184+0000 7fcd62ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd6419dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43950/0 (socket says 172.21.15.110:43950) 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.184+0000 7fcd62ffd700 1 -- 172.21.15.110:0/953705659 learned_addr learned my addr 172.21.15.110:0/953705659 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.185+0000 7fcd62ffd700 1 -- 172.21.15.110:0/953705659 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6419e160 con 0x7fcd64107f00 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.185+0000 7fcd62ffd700 1 --2- 172.21.15.110:0/953705659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd6419dc20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006af0 tx=0x7fcd54011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.185+0000 7fcd4bfff700 1 -- 172.21.15.110:0/953705659 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd54003930 con 0x7fcd64107f00 2024-04-24T23:52:07.906 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.185+0000 7fcd4bfff700 1 -- 172.21.15.110:0/953705659 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcd54003ad0 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.185+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6419e3f0 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.185+0000 7fcd4bfff700 1 -- 172.21.15.110:0/953705659 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd540195e0 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.185+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd641a10e0 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.186+0000 7fcd4bfff700 1 -- 172.21.15.110:0/953705659 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 2) ==== 46043+0+0 (secure 0 0 0) 0x7fcd540198d0 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.186+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd64063890 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.186+0000 7fcd4bfff700 1 -- 172.21.15.110:0/953705659 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd5404e0c0 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.189+0000 7fcd4bfff700 1 -- 172.21.15.110:0/953705659 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcd54023c90 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.261+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcd641a12b0 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.261+0000 7fcd4bfff700 1 -- 172.21.15.110:0/953705659 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fcd641a12b0 con 0x7fcd64107f00 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.262+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 msgr2=0x7fcd6419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.262+0000 7fcd6939a700 1 --2- 172.21.15.110:0/953705659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd6419dc20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006af0 tx=0x7fcd54011b70 comp rx=0 tx=0).stop 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.262+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 shutdown_connections 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.262+0000 7fcd6939a700 1 --2- 172.21.15.110:0/953705659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd64107f00 0x7fcd6419dc20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.262+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 >> 172.21.15.110:0/953705659 conn(0x7fcd64101630 msgr2=0x7fcd64194240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:07.907 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.262+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 shutdown_connections 2024-04-24T23:52:07.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:07.262+0000 7fcd6939a700 1 -- 172.21.15.110:0/953705659 wait complete. 2024-04-24T23:52:07.908 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (3/15)... 2024-04-24T23:52:08.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:08 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/953705659' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:52:08.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:08 smithi110 ceph-mon[29187]: mgrmap e3: x(active, since 1.00723s) 2024-04-24T23:52:10.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:10 smithi110 ceph-mon[29187]: mgrmap e4: x(active, since 2s) 2024-04-24T23:52:11.179 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:52:11.179 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:52:11.179 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:52:11.180 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:52:11.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:52:11.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:51:53.430606+0000", 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:52:11.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:52:11.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.320+0000 7fd7c82d6700 1 Processor -- start 2024-04-24T23:52:11.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.320+0000 7fd7c82d6700 1 -- start start 2024-04-24T23:52:11.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.320+0000 7fd7c82d6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c0107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:11.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.320+0000 7fd7c82d6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01084a0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.321+0000 7fd7c6072700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c0107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:11.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.321+0000 7fd7c6072700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c0107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43974/0 (socket says 172.21.15.110:43974) 2024-04-24T23:52:11.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.321+0000 7fd7c6072700 1 -- 172.21.15.110:0/166990941 learned_addr learned my addr 172.21.15.110:0/166990941 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:11.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.321+0000 7fd7c6072700 1 -- 172.21.15.110:0/166990941 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c0108c60 con 0x7fd7c0107ab0 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.321+0000 7fd7c6072700 1 --2- 172.21.15.110:0/166990941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c0107ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098a0 tx=0x7fd7bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdaf48d626f3f354 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.321+0000 7fd7c5070700 1 -- 172.21.15.110:0/166990941 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc004160 con 0x7fd7c0107ab0 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.321+0000 7fd7c5070700 1 -- 172.21.15.110:0/166990941 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd7bc004300 con 0x7fd7c0107ab0 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.321+0000 7fd7c5070700 1 -- 172.21.15.110:0/166990941 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc011a80 con 0x7fd7c0107ab0 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 -- 172.21.15.110:0/166990941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 msgr2=0x7fd7c0107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 --2- 172.21.15.110:0/166990941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c0107ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098a0 tx=0x7fd7bc009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 -- 172.21.15.110:0/166990941 shutdown_connections 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 --2- 172.21.15.110:0/166990941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c0107ed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 -- 172.21.15.110:0/166990941 >> 172.21.15.110:0/166990941 conn(0x7fd7c0103090 msgr2=0x7fd7c01054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 -- 172.21.15.110:0/166990941 shutdown_connections 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 -- 172.21.15.110:0/166990941 wait complete. 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 Processor -- start 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.322+0000 7fd7c82d6700 1 -- start start 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c82d6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:11.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c82d6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c010c010 con 0x7fd7c0107ab0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c6072700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c6072700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43976/0 (socket says 172.21.15.110:43976) 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c6072700 1 -- 172.21.15.110:0/524683636 learned_addr learned my addr 172.21.15.110:0/524683636 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c6072700 1 -- 172.21.15.110:0/524683636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c019bf60 con 0x7fd7c0107ab0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c6072700 1 --2- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c019ba20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc000c00 tx=0x7fd7bc004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7b77fe700 1 -- 172.21.15.110:0/524683636 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc0039c0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7b77fe700 1 -- 172.21.15.110:0/524683636 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd7bc003b60 con 0x7fd7c0107ab0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c019c1f0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7b77fe700 1 -- 172.21.15.110:0/524683636 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc01a6f0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.323+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c019eee0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.324+0000 7fd7b77fe700 1 -- 172.21.15.110:0/524683636 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd7bc01a9a0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.324+0000 7fd7b77fe700 1 --2- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7fd7ac039000 0x7fd7ac03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.324+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c0063890 con 0x7fd7c0107ab0 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.324+0000 7fd7b77fe700 1 -- 172.21.15.110:0/524683636 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd7bc04d4a0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.324+0000 7fd7c5871700 1 --2- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7fd7ac039000 0x7fd7ac03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.325+0000 7fd7c5871700 1 --2- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7fd7ac039000 0x7fd7ac03b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.329+0000 7fd7b77fe700 1 -- 172.21.15.110:0/524683636 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd7bc04cec0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.603+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd7c019f4b0 con 0x7fd7c0107ab0 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.604+0000 7fd7b77fe700 1 -- 172.21.15.110:0/524683636 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd7bc00bc40 con 0x7fd7c0107ab0 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7fd7ac039000 msgr2=0x7fd7ac03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 --2- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7fd7ac039000 0x7fd7ac03b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 msgr2=0x7fd7c019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 --2- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c019ba20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc000c00 tx=0x7fd7bc004290 comp rx=0 tx=0).stop 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 shutdown_connections 2024-04-24T23:52:11.187 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 --2- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7fd7ac039000 0x7fd7ac03b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:11.188 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 --2- 172.21.15.110:0/524683636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7c0107ab0 0x7fd7c019ba20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:11.188 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 >> 172.21.15.110:0/524683636 conn(0x7fd7c0103090 msgr2=0x7fd7c0103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:11.188 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 shutdown_connections 2024-04-24T23:52:11.188 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:10.606+0000 7fd7c82d6700 1 -- 172.21.15.110:0/524683636 wait complete. 2024-04-24T23:52:11.188 INFO:teuthology.orchestra.run.smithi110.stdout:mgr is available 2024-04-24T23:52:11.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:11 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/524683636' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:52:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:12 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1189178673' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout fsid = 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:52:17.789 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.605+0000 7ff7ac71a700 1 Processor -- start 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.605+0000 7ff7ac71a700 1 -- start start 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.606+0000 7ff7ac71a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a4107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:17.790 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.606+0000 7ff7ac71a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41084a0 con 0x7ff7a4107ab0 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.606+0000 7ff7aa4b6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a4107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.606+0000 7ff7aa4b6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a4107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43986/0 (socket says 172.21.15.110:43986) 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.606+0000 7ff7aa4b6700 1 -- 172.21.15.110:0/1288886591 learned_addr learned my addr 172.21.15.110:0/1288886591 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.606+0000 7ff7aa4b6700 1 -- 172.21.15.110:0/1288886591 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a4108c60 con 0x7ff7a4107ab0 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.606+0000 7ff7aa4b6700 1 --2- 172.21.15.110:0/1288886591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a4107ed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7940098a0 tx=0x7ff794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff63424b9db6aba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.606+0000 7ff7a94b4700 1 -- 172.21.15.110:0/1288886591 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff794004160 con 0x7ff7a4107ab0 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7a94b4700 1 -- 172.21.15.110:0/1288886591 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff794004300 con 0x7ff7a4107ab0 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7a94b4700 1 -- 172.21.15.110:0/1288886591 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff794011a80 con 0x7ff7a4107ab0 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1288886591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 msgr2=0x7ff7a4107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7ac71a700 1 --2- 172.21.15.110:0/1288886591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a4107ed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7940098a0 tx=0x7ff794009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1288886591 shutdown_connections 2024-04-24T23:52:17.791 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7ac71a700 1 --2- 172.21.15.110:0/1288886591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a4107ed0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1288886591 >> 172.21.15.110:0/1288886591 conn(0x7ff7a4103090 msgr2=0x7ff7a41054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1288886591 shutdown_connections 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.607+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1288886591 wait complete. 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7ac71a700 1 Processor -- start 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7ac71a700 1 -- start start 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7ac71a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a407db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7ac71a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a410c010 con 0x7ff7a4107ab0 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7aa4b6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a407db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7aa4b6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a407db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43992/0 (socket says 172.21.15.110:43992) 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7aa4b6700 1 -- 172.21.15.110:0/1189178673 learned_addr learned my addr 172.21.15.110:0/1189178673 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7aa4b6700 1 -- 172.21.15.110:0/1189178673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a407e050 con 0x7ff7a4107ab0 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.608+0000 7ff7aa4b6700 1 --2- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a407db10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff794006af0 tx=0x7ff794011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:17.792 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.609+0000 7ff79b7fe700 1 -- 172.21.15.110:0/1189178673 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff794003930 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.609+0000 7ff79b7fe700 1 -- 172.21.15.110:0/1189178673 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff794003ad0 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.609+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a407c240 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.609+0000 7ff79b7fe700 1 -- 172.21.15.110:0/1189178673 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7940195e0 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.609+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a407c780 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.609+0000 7ff79b7fe700 1 -- 172.21.15.110:0/1189178673 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff79401a430 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.610+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a4063890 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.610+0000 7ff79b7fe700 1 --2- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7ff790038fb0 0x7ff79003b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.610+0000 7ff79b7fe700 1 -- 172.21.15.110:0/1189178673 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff79404d4c0 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.610+0000 7ff7a9cb5700 1 --2- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7ff790038fb0 0x7ff79003b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.610+0000 7ff7a9cb5700 1 --2- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7ff790038fb0 0x7ff79003b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006fd0 tx=0x7ff7a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.615+0000 7ff79b7fe700 1 -- 172.21.15.110:0/1189178673 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff79404cee0 con 0x7ff7a4107ab0 2024-04-24T23:52:17.793 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.812+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff7a407cf70 con 0x7ff7a4107ab0 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.813+0000 7ff79b7fe700 1 -- 172.21.15.110:0/1189178673 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7ff794019900 con 0x7ff7a4107ab0 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7ff790038fb0 msgr2=0x7ff79003b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 --2- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7ff790038fb0 0x7ff79003b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0006fd0 tx=0x7ff7a0006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 msgr2=0x7ff7a407db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 --2- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a407db10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff794006af0 tx=0x7ff794011b70 comp rx=0 tx=0).stop 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 shutdown_connections 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 --2- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7ff790038fb0 0x7ff79003b470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 --2- 172.21.15.110:0/1189178673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7a4107ab0 0x7ff7a407db10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 >> 172.21.15.110:0/1189178673 conn(0x7ff7a4103090 msgr2=0x7ff7a4103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 shutdown_connections 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:11.815+0000 7ff7ac71a700 1 -- 172.21.15.110:0/1189178673 wait complete. 2024-04-24T23:52:17.794 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling cephadm module... 2024-04-24T23:52:21.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:21 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2808477712' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T23:52:21.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: ignoring --setuser ceph since I am not root 2024-04-24T23:52:21.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: ignoring --setgroup ceph since I am not root 2024-04-24T23:52:21.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:21.219+0000 7f451e819200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:52:21.785 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:21.402+0000 7f451e819200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:52:21.785 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:21.504+0000 7f451e819200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:52:22.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:22 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2808477712' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T23:52:22.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:22 smithi110 ceph-mon[29187]: mgrmap e5: x(active, since 13s) 2024-04-24T23:52:22.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:21.813+0000 7f451e819200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:52:22.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:21.898+0000 7f451e819200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:52:23.330 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:23.069+0000 7f451e819200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:52:23.330 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:23.239+0000 7f451e819200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:52:23.618 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:23.409+0000 7f451e819200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:52:23.620 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14b50b2700 1 Processor -- start 2024-04-24T23:52:23.621 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14b50b2700 1 -- start start 2024-04-24T23:52:23.622 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14b50b2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b0106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:23.622 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14b50b2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14b007bdf0 con 0x7f14b0106350 2024-04-24T23:52:23.622 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14aed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b0106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:23.622 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14aed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b0106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42864/0 (socket says 172.21.15.110:42864) 2024-04-24T23:52:23.622 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14aed9d700 1 -- 172.21.15.110:0/1286783849 learned_addr learned my addr 172.21.15.110:0/1286783849 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:23.622 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14aed9d700 1 -- 172.21.15.110:0/1286783849 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b0106cb0 con 0x7f14b0106350 2024-04-24T23:52:23.622 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.730+0000 7f14aed9d700 1 --2- 172.21.15.110:0/1286783849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b0106770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14a00098a0 tx=0x7f14a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cbf7ef20d64854 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:23.622 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14add9b700 1 -- 172.21.15.110:0/1286783849 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a0004160 con 0x7f14b0106350 2024-04-24T23:52:23.626 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14add9b700 1 -- 172.21.15.110:0/1286783849 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f14a0004300 con 0x7f14b0106350 2024-04-24T23:52:23.626 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14add9b700 1 -- 172.21.15.110:0/1286783849 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a0011a80 con 0x7f14b0106350 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14b50b2700 1 -- 172.21.15.110:0/1286783849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 msgr2=0x7f14b0106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14b50b2700 1 --2- 172.21.15.110:0/1286783849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b0106770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14a00098a0 tx=0x7f14a0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14b50b2700 1 -- 172.21.15.110:0/1286783849 shutdown_connections 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14b50b2700 1 --2- 172.21.15.110:0/1286783849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b0106770 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14b50b2700 1 -- 172.21.15.110:0/1286783849 >> 172.21.15.110:0/1286783849 conn(0x7f14b0101f70 msgr2=0x7f14b0104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14b50b2700 1 -- 172.21.15.110:0/1286783849 shutdown_connections 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.731+0000 7f14b50b2700 1 -- 172.21.15.110:0/1286783849 wait complete. 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.732+0000 7f14b50b2700 1 Processor -- start 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.732+0000 7f14b50b2700 1 -- start start 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.732+0000 7f14b50b2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.732+0000 7f14b50b2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0107ce0 con 0x7f14b0106350 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f14aed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:23.627 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f14aed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b019dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42876/0 (socket says 172.21.15.110:42876) 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f14aed9d700 1 -- 172.21.15.110:0/2808477712 learned_addr learned my addr 172.21.15.110:0/2808477712 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f14aed9d700 1 -- 172.21.15.110:0/2808477712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b019e150 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f14aed9d700 1 --2- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b019dc10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006af0 tx=0x7f14a0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f1497fff700 1 -- 172.21.15.110:0/2808477712 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a0003930 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f1497fff700 1 -- 172.21.15.110:0/2808477712 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f14a0003ad0 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b019e3e0 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f1497fff700 1 -- 172.21.15.110:0/2808477712 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14a00195e0 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.733+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b01a10d0 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.734+0000 7f1497fff700 1 -- 172.21.15.110:0/2808477712 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f14a00198e0 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.734+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b0063890 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.734+0000 7f1497fff700 1 --2- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f1498038fb0 0x7f149803b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.734+0000 7f1497fff700 1 -- 172.21.15.110:0/2808477712 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f14a004d530 con 0x7f14b0106350 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.734+0000 7f14ae59c700 1 --2- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f1498038fb0 0x7f149803b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.735+0000 7f14ae59c700 1 --2- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f1498038fb0 0x7f149803b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14a4006fd0 tx=0x7f14a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:23.628 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.739+0000 7f1497fff700 1 -- 172.21.15.110:0/2808477712 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14a004cf50 con 0x7f14b0106350 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:20.994+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f14b00008d0 con 0x7f14b0106350 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.045+0000 7f1497fff700 1 -- 172.21.15.110:0/2808477712 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f14a000fda0 con 0x7f14b0106350 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.048+0000 7f1497fff700 1 -- 172.21.15.110:0/2808477712 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f14a002d960 con 0x7f14b0106350 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f1498038fb0 msgr2=0x7f149803b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 --2- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f1498038fb0 0x7f149803b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14a4006fd0 tx=0x7f14a4006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 msgr2=0x7f14b019dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 --2- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b019dc10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006af0 tx=0x7f14a0011b70 comp rx=0 tx=0).stop 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 shutdown_connections 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 --2- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f1498038fb0 0x7f149803b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 --2- 172.21.15.110:0/2808477712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0106350 0x7f14b019dc10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 >> 172.21.15.110:0/2808477712 conn(0x7f14b0101f70 msgr2=0x7f14b01948b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 shutdown_connections 2024-04-24T23:52:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:21.049+0000 7f14b50b2700 1 -- 172.21.15.110:0/2808477712 wait complete. 2024-04-24T23:52:23.869 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:23.714+0000 7f451e819200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:52:23.869 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:23.812+0000 7f451e819200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:52:24.217 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:24 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:24.017+0000 7f451e819200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:52:24.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:24 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2811496732' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:52:24.536 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:24 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:24.218+0000 7f451e819200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:52:24.925 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:24 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:24.825+0000 7f451e819200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:52:24.935 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.077+0000 7f704389e700 1 Processor -- start 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.077+0000 7f704389e700 1 -- start start 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.077+0000 7f704389e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c106160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.077+0000 7f704389e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f703c106730 con 0x7f703c105d40 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.078+0000 7f704163a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c106160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.078+0000 7f704163a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c106160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42900/0 (socket says 172.21.15.110:42900) 2024-04-24T23:52:24.936 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.078+0000 7f704163a700 1 -- 172.21.15.110:0/4078026134 learned_addr learned my addr 172.21.15.110:0/4078026134 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.078+0000 7f704163a700 1 -- 172.21.15.110:0/4078026134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c108cd0 con 0x7f703c105d40 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.078+0000 7f704163a700 1 --2- 172.21.15.110:0/4078026134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c106160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f70380098a0 tx=0x7f7038009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bf8513ac773015 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.078+0000 7f7033fff700 1 -- 172.21.15.110:0/4078026134 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7038004160 con 0x7f703c105d40 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.078+0000 7f7033fff700 1 -- 172.21.15.110:0/4078026134 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7038004300 con 0x7f703c105d40 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.078+0000 7f7033fff700 1 -- 172.21.15.110:0/4078026134 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7038011a80 con 0x7f703c105d40 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.079+0000 7f704389e700 1 -- 172.21.15.110:0/4078026134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 msgr2=0x7f703c106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.079+0000 7f704389e700 1 --2- 172.21.15.110:0/4078026134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c106160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f70380098a0 tx=0x7f7038009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.079+0000 7f704389e700 1 -- 172.21.15.110:0/4078026134 shutdown_connections 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.079+0000 7f704389e700 1 --2- 172.21.15.110:0/4078026134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c106160 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.079+0000 7f704389e700 1 -- 172.21.15.110:0/4078026134 >> 172.21.15.110:0/4078026134 conn(0x7f703c101630 msgr2=0x7f703c103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:24.937 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.079+0000 7f704389e700 1 -- 172.21.15.110:0/4078026134 shutdown_connections 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.079+0000 7f704389e700 1 -- 172.21.15.110:0/4078026134 wait complete. 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.080+0000 7f704389e700 1 Processor -- start 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.080+0000 7f704389e700 1 -- start start 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.080+0000 7f704389e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.080+0000 7f704389e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f703c10c080 con 0x7f703c105d40 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.080+0000 7f704163a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.080+0000 7f704163a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42906/0 (socket says 172.21.15.110:42906) 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.080+0000 7f704163a700 1 -- 172.21.15.110:0/2811496732 learned_addr learned my addr 172.21.15.110:0/2811496732 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.081+0000 7f704163a700 1 -- 172.21.15.110:0/2811496732 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c19bf70 con 0x7f703c105d40 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.081+0000 7f704163a700 1 --2- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c19ba30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7038000c00 tx=0x7f7038004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.081+0000 7f70327fc700 1 -- 172.21.15.110:0/2811496732 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70380039c0 con 0x7f703c105d40 2024-04-24T23:52:24.938 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.081+0000 7f70327fc700 1 -- 172.21.15.110:0/2811496732 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7038003b60 con 0x7f703c105d40 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.081+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c19c200 con 0x7f703c105d40 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.081+0000 7f70327fc700 1 -- 172.21.15.110:0/2811496732 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f703801a6f0 con 0x7f703c105d40 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.081+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c19eef0 con 0x7f703c105d40 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.082+0000 7f70327fc700 1 -- 172.21.15.110:0/2811496732 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f703801a9a0 con 0x7f703c105d40 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.082+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c063890 con 0x7f703c105d40 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.082+0000 7f70327fc700 1 --2- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f7028039050 0x7f702803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.082+0000 7f70327fc700 1 -- 172.21.15.110:0/2811496732 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f703804d2a0 con 0x7f703c105d40 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.082+0000 7f7040e39700 1 -- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f7028039050 msgr2=0x7f702803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/2433943234 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.082+0000 7f7040e39700 1 --2- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f7028039050 0x7f702803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.091+0000 7f70327fc700 1 -- 172.21.15.110:0/2811496732 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7038020240 con 0x7f703c105d40 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.283+0000 7f7040e39700 1 -- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f7028039050 msgr2=0x7f702803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/2433943234 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.283+0000 7f7040e39700 1 --2- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f7028039050 0x7f702803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:52:24.939 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.355+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f703c19f0c0 con 0x7f703c105d40 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.355+0000 7f70327fc700 1 -- 172.21.15.110:0/2811496732 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f703c19f0c0 con 0x7f703c105d40 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.357+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f7028039050 msgr2=0x7f702803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.357+0000 7f704389e700 1 --2- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f7028039050 0x7f702803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.357+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 msgr2=0x7f703c19ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.357+0000 7f704389e700 1 --2- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c19ba30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7038000c00 tx=0x7f7038004290 comp rx=0 tx=0).stop 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.358+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 shutdown_connections 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.358+0000 7f704389e700 1 --2- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7f7028039050 0x7f702803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.358+0000 7f704389e700 1 --2- 172.21.15.110:0/2811496732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f703c105d40 0x7f703c19ba30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.358+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 >> 172.21.15.110:0/2811496732 conn(0x7f703c101630 msgr2=0x7f703c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.358+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 shutdown_connections 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:24.358+0000 7f704389e700 1 -- 172.21.15.110:0/2811496732 wait complete. 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for the mgr to restart... 2024-04-24T23:52:24.940 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr epoch 5... 2024-04-24T23:52:25.535 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:25 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:25.260+0000 7f451e819200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:52:25.536 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:25 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:25.361+0000 7f451e819200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:52:25.940 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:25 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:25.561+0000 7f451e819200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:52:26.285 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:26.081+0000 7f451e819200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:52:26.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:26.172+0000 7f451e819200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:52:26.741 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:26.412+0000 7f451e819200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:52:26.742 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:26.550+0000 7f451e819200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:52:26.742 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:26.648+0000 7f451e819200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:52:27.035 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:26.762+0000 7f451e819200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:52:27.036 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:26.852+0000 7f451e819200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:52:27.915 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:27.650+0000 7f451e819200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:52:27.915 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:27.863+0000 7f451e819200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:52:28.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: Active manager daemon x restarted 2024-04-24T23:52:28.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: Activating manager daemon x 2024-04-24T23:52:28.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T23:52:28.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: mgrmap e6: x(active, starting, since 0.00327666s) 2024-04-24T23:52:28.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:52:28.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: Manager daemon x is now available 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:52:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:27 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:52:29.225 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:28 smithi110 ceph-mon[29187]: Found migration_current of "None". Setting to last migration. 2024-04-24T23:52:29.225 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:28 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:29.225 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:28 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:29.225 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:28 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:52:29.225 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:28 smithi110 ceph-mon[29187]: mgrmap e7: x(active, since 1.00573s) 2024-04-24T23:52:29.484 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.335+0000 7faa3b75b700 1 Processor -- start 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.335+0000 7faa3b75b700 1 -- start start 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.335+0000 7faa3b75b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3410a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.335+0000 7faa3b75b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faa3407bd90 con 0x7faa34108540 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.336+0000 7faa394f7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3410a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.336+0000 7faa394f7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3410a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42922/0 (socket says 172.21.15.110:42922) 2024-04-24T23:52:29.485 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.336+0000 7faa394f7700 1 -- 172.21.15.110:0/1060669025 learned_addr learned my addr 172.21.15.110:0/1060669025 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:29.486 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.336+0000 7faa394f7700 1 -- 172.21.15.110:0/1060669025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3410ae70 con 0x7faa34108540 2024-04-24T23:52:29.486 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.336+0000 7faa394f7700 1 --2- 172.21.15.110:0/1060669025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3410a930 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa240098a0 tx=0x7faa24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ced37223ffe0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:29.486 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.336+0000 7faa2bfff700 1 -- 172.21.15.110:0/1060669025 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa24004160 con 0x7faa34108540 2024-04-24T23:52:29.486 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.336+0000 7faa2bfff700 1 -- 172.21.15.110:0/1060669025 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7faa24004300 con 0x7faa34108540 2024-04-24T23:52:29.486 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.336+0000 7faa2bfff700 1 -- 172.21.15.110:0/1060669025 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa24011a80 con 0x7faa34108540 2024-04-24T23:52:29.486 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 -- 172.21.15.110:0/1060669025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 msgr2=0x7faa3410a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 --2- 172.21.15.110:0/1060669025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3410a930 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa240098a0 tx=0x7faa24009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 -- 172.21.15.110:0/1060669025 shutdown_connections 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 --2- 172.21.15.110:0/1060669025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3410a930 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 -- 172.21.15.110:0/1060669025 >> 172.21.15.110:0/1060669025 conn(0x7faa34101eb0 msgr2=0x7faa341042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 -- 172.21.15.110:0/1060669025 shutdown_connections 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 -- 172.21.15.110:0/1060669025 wait complete. 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 Processor -- start 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.337+0000 7faa3b75b700 1 -- start start 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa3b75b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa3b75b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faa3410bd80 con 0x7faa34108540 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa394f7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:29.487 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa394f7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42924/0 (socket says 172.21.15.110:42924) 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa394f7700 1 -- 172.21.15.110:0/1812014112 learned_addr learned my addr 172.21.15.110:0/1812014112 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa394f7700 1 -- 172.21.15.110:0/1812014112 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3419bef0 con 0x7faa34108540 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa394f7700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3419b9b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa24006af0 tx=0x7faa24011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa24003930 con 0x7faa34108540 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7faa24003ad0 con 0x7faa34108540 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa240195e0 con 0x7faa34108540 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3419c180 con 0x7faa34108540 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.338+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3419ee70 con 0x7faa34108540 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.339+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7faa2401a430 con 0x7faa34108540 2024-04-24T23:52:29.488 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.339+0000 7faa2a7fc700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 0x7faa2003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.339+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 --> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faa2003bbd0 con 0x7faa20039000 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.339+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faa2404d580 con 0x7faa34108540 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.339+0000 7faa38cf6700 1 -- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 msgr2=0x7faa2003b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/2433943234 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.339+0000 7faa38cf6700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 0x7faa2003b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.540+0000 7faa38cf6700 1 -- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 msgr2=0x7faa2003b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/2433943234 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.540+0000 7faa38cf6700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 0x7faa2003b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.941+0000 7faa38cf6700 1 -- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 msgr2=0x7faa2003b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/2433943234 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:25.941+0000 7faa38cf6700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 0x7faa2003b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:52:29.489 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:26.742+0000 7faa38cf6700 1 -- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 msgr2=0x7faa2003b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/2433943234 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:26.742+0000 7faa38cf6700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 0x7faa2003b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:27.866+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7faa2404c040 con 0x7faa34108540 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:27.866+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 msgr2=0x7faa2003b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:27.866+0000 7faa2a7fc700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 0x7faa2003b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.867+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7faa24023390 con 0x7faa34108540 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.867+0000 7faa2a7fc700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7faa2003c930 0x7faa2003ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.867+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faa2402de30 con 0x7faa2003c930 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.868+0000 7faa38cf6700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7faa2003c930 0x7faa2003ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.869+0000 7faa38cf6700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7faa2003c930 0x7faa2003ed20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faa30003a10 tx=0x7faa30009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:29.490 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.869+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mgr.14120 v2:172.21.15.110:6800/1818468115 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7faa2402de30 con 0x7faa2003c930 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.876+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7faa34063890 con 0x7faa2003c930 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.876+0000 7faa2a7fc700 1 -- 172.21.15.110:0/1812014112 <== mgr.14120 v2:172.21.15.110:6800/1818468115 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7faa34063890 con 0x7faa2003c930 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7faa2003c930 msgr2=0x7faa2003ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7faa2003c930 0x7faa2003ed20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faa30003a10 tx=0x7faa30009290 comp rx=0 tx=0).stop 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 msgr2=0x7faa3419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3419b9b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa24006af0 tx=0x7faa24011b70 comp rx=0 tx=0).stop 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 shutdown_connections 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7faa2003c930 0x7faa2003ed20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa34108540 0x7faa3419b9b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 --2- 172.21.15.110:0/1812014112 >> [v2:172.21.15.110:6800/2433943234,v1:172.21.15.110:6801/2433943234] conn(0x7faa20039000 0x7faa2003b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:29.491 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 >> 172.21.15.110:0/1812014112 conn(0x7faa34101eb0 msgr2=0x7faa34102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:29.492 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 shutdown_connections 2024-04-24T23:52:29.492 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:28.877+0000 7faa3b75b700 1 -- 172.21.15.110:0/1812014112 wait complete. 2024-04-24T23:52:29.492 INFO:teuthology.orchestra.run.smithi110.stdout:mgr epoch 5 is available 2024-04-24T23:52:29.492 INFO:teuthology.orchestra.run.smithi110.stdout:Setting orchestrator backend to cephadm... 2024-04-24T23:52:30.139 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:29 smithi110 ceph-mon[29187]: [24/Apr/2024:23:52:28] ENGINE Bus STARTING 2024-04-24T23:52:30.139 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:29 smithi110 ceph-mon[29187]: [24/Apr/2024:23:52:28] ENGINE Serving on https://172.21.15.110:7150 2024-04-24T23:52:30.139 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:29 smithi110 ceph-mon[29187]: [24/Apr/2024:23:52:28] ENGINE Serving on http://172.21.15.110:8765 2024-04-24T23:52:30.139 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:29 smithi110 ceph-mon[29187]: [24/Apr/2024:23:52:28] ENGINE Bus STARTED 2024-04-24T23:52:30.139 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:29 smithi110 ceph-mon[29187]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:52:30.140 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:29 smithi110 ceph-mon[29187]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:52:30.732 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.915+0000 7f6a45fcd700 1 Processor -- start 2024-04-24T23:52:30.738 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.915+0000 7f6a45fcd700 1 -- start start 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.915+0000 7f6a45fcd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a40106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.915+0000 7f6a45fcd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4007bdf0 con 0x7f6a40106350 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.915+0000 7f6a3f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a40106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.915+0000 7f6a3f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a40106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54576/0 (socket says 172.21.15.110:54576) 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.915+0000 7f6a3f7fe700 1 -- 172.21.15.110:0/1974967992 learned_addr learned my addr 172.21.15.110:0/1974967992 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.916+0000 7f6a3f7fe700 1 -- 172.21.15.110:0/1974967992 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a40106cb0 con 0x7f6a40106350 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.916+0000 7f6a3f7fe700 1 --2- 172.21.15.110:0/1974967992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a40106770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a300098a0 tx=0x7f6a30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed301c352e5b0bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.916+0000 7f6a3e7fc700 1 -- 172.21.15.110:0/1974967992 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a30004160 con 0x7f6a40106350 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.916+0000 7f6a3e7fc700 1 -- 172.21.15.110:0/1974967992 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6a30004300 con 0x7f6a40106350 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.916+0000 7f6a3e7fc700 1 -- 172.21.15.110:0/1974967992 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a30011a80 con 0x7f6a40106350 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.917+0000 7f6a45fcd700 1 -- 172.21.15.110:0/1974967992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 msgr2=0x7f6a40106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:30.739 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.917+0000 7f6a45fcd700 1 --2- 172.21.15.110:0/1974967992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a40106770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a300098a0 tx=0x7f6a30009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.917+0000 7f6a45fcd700 1 -- 172.21.15.110:0/1974967992 shutdown_connections 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.917+0000 7f6a45fcd700 1 --2- 172.21.15.110:0/1974967992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a40106770 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.917+0000 7f6a45fcd700 1 -- 172.21.15.110:0/1974967992 >> 172.21.15.110:0/1974967992 conn(0x7f6a40101f70 msgr2=0x7f6a40104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.917+0000 7f6a45fcd700 1 -- 172.21.15.110:0/1974967992 shutdown_connections 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.917+0000 7f6a45fcd700 1 -- 172.21.15.110:0/1974967992 wait complete. 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.917+0000 7f6a45fcd700 1 Processor -- start 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a45fcd700 1 -- start start 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a45fcd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a4019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a45fcd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a40107ce0 con 0x7f6a40106350 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a3f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a4019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a3f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a4019dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54592/0 (socket says 172.21.15.110:54592) 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a3f7fe700 1 -- 172.21.15.110:0/733501599 learned_addr learned my addr 172.21.15.110:0/733501599 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a3f7fe700 1 -- 172.21.15.110:0/733501599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4019e150 con 0x7f6a40106350 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a3f7fe700 1 --2- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a4019dc10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a30011a50 tx=0x7f6a30004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a3cff9700 1 -- 172.21.15.110:0/733501599 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a300039f0 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a3cff9700 1 -- 172.21.15.110:0/733501599 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6a30003b90 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4019e3e0 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a3cff9700 1 -- 172.21.15.110:0/733501599 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a3001a600 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.918+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a401a10d0 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.919+0000 7f6a3cff9700 1 -- 172.21.15.110:0/733501599 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f6a30024440 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.919+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a40063890 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.919+0000 7f6a3cff9700 1 --2- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f6a28038f30 0x7f6a2803b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.920+0000 7f6a3cff9700 1 -- 172.21.15.110:0/733501599 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a3004d310 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.920+0000 7f6a3effd700 1 --2- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f6a28038f30 0x7f6a2803b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.920+0000 7f6a3effd700 1 --2- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f6a28038f30 0x7f6a2803b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a34006fd0 tx=0x7f6a34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.925+0000 7f6a3cff9700 1 -- 172.21.15.110:0/733501599 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6a3000f2e0 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:29.990+0000 7f6a3cff9700 1 -- 172.21.15.110:0/733501599 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6a30024a40 con 0x7f6a40106350 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.139+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6a400008d0 con 0x7f6a28038f30 2024-04-24T23:52:30.741 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.144+0000 7f6a3cff9700 1 -- 172.21.15.110:0/733501599 <== mgr.14120 v2:172.21.15.110:6800/1818468115 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a400008d0 con 0x7f6a28038f30 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f6a28038f30 msgr2=0x7f6a2803b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 --2- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f6a28038f30 0x7f6a2803b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a34006fd0 tx=0x7f6a34006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 msgr2=0x7f6a4019dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 --2- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a4019dc10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a30011a50 tx=0x7f6a30004290 comp rx=0 tx=0).stop 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 shutdown_connections 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 --2- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f6a28038f30 0x7f6a2803b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 --2- 172.21.15.110:0/733501599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a40106350 0x7f6a4019dc10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 >> 172.21.15.110:0/733501599 conn(0x7f6a40101f70 msgr2=0x7f6a401948b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 shutdown_connections 2024-04-24T23:52:30.742 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:30.146+0000 7f6a45fcd700 1 -- 172.21.15.110:0/733501599 wait complete. 2024-04-24T23:52:31.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:30 smithi110 ceph-mon[29187]: mgrmap e8: x(active, since 2s) 2024-04-24T23:52:31.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:30 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:31.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:30 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:52:31.961 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T23:52:31.961 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.147+0000 7f12323a3700 1 Processor -- start 2024-04-24T23:52:31.961 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.147+0000 7f12323a3700 1 -- start start 2024-04-24T23:52:31.961 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.147+0000 7f12323a3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f12323a3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f122c1084c0 con 0x7f122c107ad0 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f122bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f122bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c107ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54606/0 (socket says 172.21.15.110:54606) 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f122bfff700 1 -- 172.21.15.110:0/839425113 learned_addr learned my addr 172.21.15.110:0/839425113 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f122bfff700 1 -- 172.21.15.110:0/839425113 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c108c80 con 0x7f122c107ad0 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f122bfff700 1 --2- 172.21.15.110:0/839425113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c107ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f12200098a0 tx=0x7f1220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79675655f90ebba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f122affd700 1 -- 172.21.15.110:0/839425113 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1220004160 con 0x7f122c107ad0 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f122affd700 1 -- 172.21.15.110:0/839425113 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1220004300 con 0x7f122c107ad0 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.148+0000 7f122affd700 1 -- 172.21.15.110:0/839425113 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1220011a80 con 0x7f122c107ad0 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.149+0000 7f12323a3700 1 -- 172.21.15.110:0/839425113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 msgr2=0x7f122c107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.149+0000 7f12323a3700 1 --2- 172.21.15.110:0/839425113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c107ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f12200098a0 tx=0x7f1220009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.149+0000 7f12323a3700 1 -- 172.21.15.110:0/839425113 shutdown_connections 2024-04-24T23:52:31.962 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.149+0000 7f12323a3700 1 --2- 172.21.15.110:0/839425113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c107ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:31.965 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.149+0000 7f12323a3700 1 -- 172.21.15.110:0/839425113 >> 172.21.15.110:0/839425113 conn(0x7f122c103090 msgr2=0x7f122c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:31.965 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.149+0000 7f12323a3700 1 -- 172.21.15.110:0/839425113 shutdown_connections 2024-04-24T23:52:31.965 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.149+0000 7f12323a3700 1 -- 172.21.15.110:0/839425113 wait complete. 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.149+0000 7f12323a3700 1 Processor -- start 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.150+0000 7f12323a3700 1 -- start start 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.150+0000 7f12323a3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.150+0000 7f12323a3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f122c10c030 con 0x7f122c107ad0 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.150+0000 7f122bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.150+0000 7f122bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54620/0 (socket says 172.21.15.110:54620) 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.150+0000 7f122bfff700 1 -- 172.21.15.110:0/3554748218 learned_addr learned my addr 172.21.15.110:0/3554748218 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.150+0000 7f122bfff700 1 -- 172.21.15.110:0/3554748218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c19bf70 con 0x7f122c107ad0 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.150+0000 7f122bfff700 1 --2- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c19ba30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1220000c00 tx=0x7f1220004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.151+0000 7f12297fa700 1 -- 172.21.15.110:0/3554748218 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12200039c0 con 0x7f122c107ad0 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.151+0000 7f12297fa700 1 -- 172.21.15.110:0/3554748218 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1220003b60 con 0x7f122c107ad0 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.151+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c19c200 con 0x7f122c107ad0 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.151+0000 7f12297fa700 1 -- 172.21.15.110:0/3554748218 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f122001a6f0 con 0x7f122c107ad0 2024-04-24T23:52:31.966 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.151+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c19eef0 con 0x7f122c107ad0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.151+0000 7f12297fa700 1 -- 172.21.15.110:0/3554748218 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f122001a890 con 0x7f122c107ad0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.152+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c063890 con 0x7f122c107ad0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.152+0000 7f12297fa700 1 --2- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f12140390a0 0x7f121403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.152+0000 7f12297fa700 1 -- 172.21.15.110:0/3554748218 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f122004d310 con 0x7f122c107ad0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.152+0000 7f122b7fe700 1 --2- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f12140390a0 0x7f121403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.152+0000 7f122b7fe700 1 --2- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f12140390a0 0x7f121403b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f121c006fd0 tx=0x7f121c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.157+0000 7f12297fa700 1 -- 172.21.15.110:0/3554748218 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1220020240 con 0x7f122c107ad0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.353+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f122c104300 con 0x7f12140390a0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.354+0000 7f12297fa700 1 -- 172.21.15.110:0/3554748218 <== mgr.14120 v2:172.21.15.110:6800/1818468115 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f122c104300 con 0x7f12140390a0 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f12140390a0 msgr2=0x7f121403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 --2- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f12140390a0 0x7f121403b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f121c006fd0 tx=0x7f121c006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 msgr2=0x7f122c19ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 --2- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c19ba30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1220000c00 tx=0x7f1220004290 comp rx=0 tx=0).stop 2024-04-24T23:52:31.967 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 shutdown_connections 2024-04-24T23:52:31.968 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 --2- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f12140390a0 0x7f121403b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:31.968 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 --2- 172.21.15.110:0/3554748218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f122c107ad0 0x7f122c19ba30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:31.968 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 >> 172.21.15.110:0/3554748218 conn(0x7f122c103090 msgr2=0x7f122c103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:31.968 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 shutdown_connections 2024-04-24T23:52:31.968 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:31.356+0000 7f12323a3700 1 -- 172.21.15.110:0/3554748218 wait complete. 2024-04-24T23:52:31.968 INFO:teuthology.orchestra.run.smithi110.stdout:Generating ssh key... 2024-04-24T23:52:32.204 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:31 smithi110 ceph-mon[29187]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: Generating public/private rsa key pair. 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: Your identification has been saved in /tmp/tmpz84haa10/key. 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: Your public key has been saved in /tmp/tmpz84haa10/key.pub. 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: The key fingerprint is: 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: SHA256:HHdus4iY4HL7xh/OLgc+4KnJEPwf9I0YLxDPSzlr4dc ceph-787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: The key's randomart image is: 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: +---[RSA 3072]----+ 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: | | 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: | | 2024-04-24T23:52:32.991 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: | . . . . | 2024-04-24T23:52:32.992 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: |. + .. o o | 2024-04-24T23:52:32.992 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: |.. ..X S + | 2024-04-24T23:52:32.992 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: | ...*.@o+. o o | 2024-04-24T23:52:32.992 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: |. .oo&o*oE. . | 2024-04-24T23:52:32.992 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: | o +=.Xo.. | 2024-04-24T23:52:32.992 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: | +..+.*= | 2024-04-24T23:52:32.992 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: +----[SHA256]-----+ 2024-04-24T23:52:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-mon[29187]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:32 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.412+0000 7f58f9469700 1 Processor -- start 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.412+0000 7f58f9469700 1 -- start start 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.412+0000 7f58f9469700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f410a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.412+0000 7f58f9469700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58f407bd70 con 0x7f58f41085b0 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.412+0000 7f58f2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f410a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.412+0000 7f58f2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f410a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54634/0 (socket says 172.21.15.110:54634) 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.412+0000 7f58f2ffd700 1 -- 172.21.15.110:0/689284437 learned_addr learned my addr 172.21.15.110:0/689284437 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.412+0000 7f58f2ffd700 1 -- 172.21.15.110:0/689284437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f410aee0 con 0x7f58f41085b0 2024-04-24T23:52:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f2ffd700 1 --2- 172.21.15.110:0/689284437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f410a9a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f58e40098a0 tx=0x7f58e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449895f868da8ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f1ffb700 1 -- 172.21.15.110:0/689284437 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58e4004160 con 0x7f58f41085b0 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f1ffb700 1 -- 172.21.15.110:0/689284437 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f58e4004300 con 0x7f58f41085b0 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f1ffb700 1 -- 172.21.15.110:0/689284437 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58e4011a80 con 0x7f58f41085b0 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f9469700 1 -- 172.21.15.110:0/689284437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 msgr2=0x7f58f410a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f9469700 1 --2- 172.21.15.110:0/689284437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f410a9a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f58e40098a0 tx=0x7f58e4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f9469700 1 -- 172.21.15.110:0/689284437 shutdown_connections 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f9469700 1 --2- 172.21.15.110:0/689284437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f410a9a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f9469700 1 -- 172.21.15.110:0/689284437 >> 172.21.15.110:0/689284437 conn(0x7f58f4101f40 msgr2=0x7f58f4104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f9469700 1 -- 172.21.15.110:0/689284437 shutdown_connections 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.413+0000 7f58f9469700 1 -- 172.21.15.110:0/689284437 wait complete. 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.414+0000 7f58f9469700 1 Processor -- start 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.414+0000 7f58f9469700 1 -- start start 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.414+0000 7f58f9469700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f4107e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:33.318 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.414+0000 7f58f9469700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58f410bde0 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.414+0000 7f58f2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f4107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.414+0000 7f58f2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f4107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54648/0 (socket says 172.21.15.110:54648) 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.414+0000 7f58f2ffd700 1 -- 172.21.15.110:0/3371066466 learned_addr learned my addr 172.21.15.110:0/3371066466 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.415+0000 7f58f2ffd700 1 -- 172.21.15.110:0/3371066466 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f4108370 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.415+0000 7f58f2ffd700 1 --2- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f4107e30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006af0 tx=0x7f58e4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.415+0000 7f58dbfff700 1 -- 172.21.15.110:0/3371066466 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58e4003930 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.415+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f4106560 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.415+0000 7f58dbfff700 1 -- 172.21.15.110:0/3371066466 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f58e4003ad0 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.415+0000 7f58dbfff700 1 -- 172.21.15.110:0/3371066466 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58e40195e0 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.415+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f4106aa0 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.416+0000 7f58dbfff700 1 -- 172.21.15.110:0/3371066466 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f58e40198e0 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.416+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f4063890 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.416+0000 7f58dbfff700 1 --2- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f58dc039000 0x7f58dc03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.416+0000 7f58dbfff700 1 -- 172.21.15.110:0/3371066466 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f58e404d6e0 con 0x7f58f41085b0 2024-04-24T23:52:33.319 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.416+0000 7f58f27fc700 1 --2- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f58dc039000 0x7f58dc03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.417+0000 7f58f27fc700 1 --2- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f58dc039000 0x7f58dc03b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f58e8006fd0 tx=0x7f58e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.421+0000 7f58dbfff700 1 -- 172.21.15.110:0/3371066466 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f58e402d960 con 0x7f58f41085b0 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.619+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f58f41031e0 con 0x7f58dc039000 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.744+0000 7f58dbfff700 1 -- 172.21.15.110:0/3371066466 <== mgr.14120 v2:172.21.15.110:6800/1818468115 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f58f41031e0 con 0x7f58dc039000 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f58dc039000 msgr2=0x7f58dc03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 --2- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f58dc039000 0x7f58dc03b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f58e8006fd0 tx=0x7f58e8006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 msgr2=0x7f58f4107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 --2- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f4107e30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006af0 tx=0x7f58e4011b70 comp rx=0 tx=0).stop 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 shutdown_connections 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 --2- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f58dc039000 0x7f58dc03b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 --2- 172.21.15.110:0/3371066466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58f41085b0 0x7f58f4107e30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 >> 172.21.15.110:0/3371066466 conn(0x7f58f4101f40 msgr2=0x7f58f4102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 shutdown_connections 2024-04-24T23:52:33.320 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:32.746+0000 7f58f9469700 1 -- 172.21.15.110:0/3371066466 wait complete. 2024-04-24T23:52:34.266 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:33 smithi110 ceph-mon[29187]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:34.266 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:33 smithi110 ceph-mon[29187]: Generating ssh key... 2024-04-24T23:52:34.533 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAk9FA9DPlXj9C2FfL4uZ8rYP8dSuFhY1lSIhKbf87qgN28dpn7yOjGsuGbJGpG4vOvepYAcTvCGLOA5LTPHsDz42fXZZ2RJk9yw+1VdGmSywNbcNRi7drW2S7hvLmqLnboBovq25KNUbHuPf4vdw7QRb4qa54FxVZAboDcj+kY++AYcpNR0Gt0pFZ/MQLQB+Bwyju5MEhR250K3w4fD+8iAP+OyfGV9VnF7PWXrCsobOMe4F6qJbADZO5CbK3t4SDp+p2tUkBo4KBhBFWXIddhi7nTSMiXmn5Y6HUf4LkFH727sLGvdaLEai54dbmcrGn5nwHlFbufaHFTgj0Lknu1KmaK+YHcmrOsiGKi7Yja7+v9TqnM03Ona94oIl/ix/Pqb9dDkEopHR8fZi0xhl6ju/4W3Z5oHc9J8+HD0s7jTI6wCrl54g711g+E43V8RdvOoJxZiFWkpTWCepFwFINTuSZPA87jd+AH9uUmOe/uz2HaH2i8IfCJPNWHM9mUrc= ceph-787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.752+0000 7fa5bd29d700 1 Processor -- start 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.752+0000 7fa5bd29d700 1 -- start start 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.752+0000 7fa5bd29d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b8107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.752+0000 7fa5bd29d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81084a0 con 0x7fa5b8107ab0 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.753+0000 7fa5b6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b8107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.753+0000 7fa5b6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b8107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54664/0 (socket says 172.21.15.110:54664) 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.753+0000 7fa5b6ffd700 1 -- 172.21.15.110:0/3064826344 learned_addr learned my addr 172.21.15.110:0/3064826344 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.753+0000 7fa5b6ffd700 1 -- 172.21.15.110:0/3064826344 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b8108c60 con 0x7fa5b8107ab0 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.753+0000 7fa5b6ffd700 1 --2- 172.21.15.110:0/3064826344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b8107ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0098a0 tx=0x7fa5ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7f3cb2aedaa6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.753+0000 7fa5b5ffb700 1 -- 172.21.15.110:0/3064826344 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5ac004160 con 0x7fa5b8107ab0 2024-04-24T23:52:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.753+0000 7fa5b5ffb700 1 -- 172.21.15.110:0/3064826344 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa5ac004300 con 0x7fa5b8107ab0 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.753+0000 7fa5b5ffb700 1 -- 172.21.15.110:0/3064826344 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5ac011a80 con 0x7fa5b8107ab0 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.754+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3064826344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 msgr2=0x7fa5b8107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.754+0000 7fa5bd29d700 1 --2- 172.21.15.110:0/3064826344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b8107ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0098a0 tx=0x7fa5ac009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.754+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3064826344 shutdown_connections 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.754+0000 7fa5bd29d700 1 --2- 172.21.15.110:0/3064826344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b8107ed0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.754+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3064826344 >> 172.21.15.110:0/3064826344 conn(0x7fa5b8103090 msgr2=0x7fa5b81054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.754+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3064826344 shutdown_connections 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.754+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3064826344 wait complete. 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.754+0000 7fa5bd29d700 1 Processor -- start 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5bd29d700 1 -- start start 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5bd29d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:34.535 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5bd29d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810c010 con 0x7fa5b8107ab0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5b6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5b6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54680/0 (socket says 172.21.15.110:54680) 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5b6ffd700 1 -- 172.21.15.110:0/3675521249 learned_addr learned my addr 172.21.15.110:0/3675521249 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5b6ffd700 1 -- 172.21.15.110:0/3675521249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b819bf50 con 0x7fa5b8107ab0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5b6ffd700 1 --2- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b819ba10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac000c00 tx=0x7fa5ac004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa59ffff700 1 -- 172.21.15.110:0/3675521249 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5ac0039c0 con 0x7fa5b8107ab0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa59ffff700 1 -- 172.21.15.110:0/3675521249 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa5ac003b60 con 0x7fa5b8107ab0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa59ffff700 1 -- 172.21.15.110:0/3675521249 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5ac01a6f0 con 0x7fa5b8107ab0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.755+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b819c1e0 con 0x7fa5b8107ab0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.756+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b819eed0 con 0x7fa5b8107ab0 2024-04-24T23:52:34.536 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.756+0000 7fa59ffff700 1 -- 172.21.15.110:0/3675521249 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa5ac01a9a0 con 0x7fa5b8107ab0 2024-04-24T23:52:34.537 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.757+0000 7fa59ffff700 1 --2- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fa5a0039050 0x7fa5a003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:34.537 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.757+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b8063890 con 0x7fa5b8107ab0 2024-04-24T23:52:34.537 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.757+0000 7fa59ffff700 1 -- 172.21.15.110:0/3675521249 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa5ac04d4e0 con 0x7fa5b8107ab0 2024-04-24T23:52:34.537 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.757+0000 7fa5b67fc700 1 --2- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fa5a0039050 0x7fa5a003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:34.537 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.757+0000 7fa5b67fc700 1 --2- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fa5a0039050 0x7fa5a003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8006fd0 tx=0x7fa5a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:34.537 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.762+0000 7fa59ffff700 1 -- 172.21.15.110:0/3675521249 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5ac024780 con 0x7fa5b8107ab0 2024-04-24T23:52:34.537 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.959+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fa5b8104350 con 0x7fa5a0039050 2024-04-24T23:52:34.537 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.960+0000 7fa59ffff700 1 -- 172.21.15.110:0/3675521249 <== mgr.14120 v2:172.21.15.110:6800/1818468115 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fa5b8104350 con 0x7fa5a0039050 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fa5a0039050 msgr2=0x7fa5a003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 --2- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fa5a0039050 0x7fa5a003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8006fd0 tx=0x7fa5a8006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 msgr2=0x7fa5b819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 --2- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b819ba10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac000c00 tx=0x7fa5ac004290 comp rx=0 tx=0).stop 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 shutdown_connections 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 --2- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fa5a0039050 0x7fa5a003b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 --2- 172.21.15.110:0/3675521249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5b8107ab0 0x7fa5b819ba10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 >> 172.21.15.110:0/3675521249 conn(0x7fa5b8103090 msgr2=0x7fa5b8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 shutdown_connections 2024-04-24T23:52:34.538 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:33.962+0000 7fa5bd29d700 1 -- 172.21.15.110:0/3675521249 wait complete. 2024-04-24T23:52:34.539 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:52:34.539 INFO:teuthology.orchestra.run.smithi110.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T23:52:34.539 INFO:teuthology.orchestra.run.smithi110.stdout:Adding host smithi110... 2024-04-24T23:52:35.153 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:34 smithi110 ceph-mon[29187]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:36.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:36 smithi110 ceph-mon[29187]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "addr": "172.21.15.110", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:36.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:36 smithi110 ceph-mon[29187]: Deploying cephadm binary to smithi110 2024-04-24T23:52:39.749 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Added host 'smithi110' with addr '172.21.15.110' 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.945+0000 7f2f748dd700 1 Processor -- start 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.945+0000 7f2f748dd700 1 -- start start 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.945+0000 7f2f748dd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f70107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.945+0000 7f2f748dd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701084a0 con 0x7f2f70107ab0 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.945+0000 7f2f6e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f70107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.945+0000 7f2f6e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f70107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54688/0 (socket says 172.21.15.110:54688) 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.945+0000 7f2f6e59c700 1 -- 172.21.15.110:0/141701582 learned_addr learned my addr 172.21.15.110:0/141701582 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.946+0000 7f2f6e59c700 1 -- 172.21.15.110:0/141701582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f70108c60 con 0x7f2f70107ab0 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.946+0000 7f2f6e59c700 1 --2- 172.21.15.110:0/141701582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f70107ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f600098a0 tx=0x7f2f60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab114218c7d2613e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.946+0000 7f2f6d59a700 1 -- 172.21.15.110:0/141701582 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f60004160 con 0x7f2f70107ab0 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.946+0000 7f2f6d59a700 1 -- 172.21.15.110:0/141701582 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2f60004300 con 0x7f2f70107ab0 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.946+0000 7f2f6d59a700 1 -- 172.21.15.110:0/141701582 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f60011a80 con 0x7f2f70107ab0 2024-04-24T23:52:39.750 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.946+0000 7f2f748dd700 1 -- 172.21.15.110:0/141701582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 msgr2=0x7f2f70107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.946+0000 7f2f748dd700 1 --2- 172.21.15.110:0/141701582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f70107ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f600098a0 tx=0x7f2f60009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 -- 172.21.15.110:0/141701582 shutdown_connections 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 --2- 172.21.15.110:0/141701582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f70107ed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 -- 172.21.15.110:0/141701582 >> 172.21.15.110:0/141701582 conn(0x7f2f70103090 msgr2=0x7f2f701054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 -- 172.21.15.110:0/141701582 shutdown_connections 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 -- 172.21.15.110:0/141701582 wait complete. 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 Processor -- start 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 -- start start 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f7019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.947+0000 7f2f748dd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7010c010 con 0x7f2f70107ab0 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f6e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f7019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f6e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f7019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54702/0 (socket says 172.21.15.110:54702) 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f6e59c700 1 -- 172.21.15.110:0/2539336163 learned_addr learned my addr 172.21.15.110:0/2539336163 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:39.751 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f6e59c700 1 -- 172.21.15.110:0/2539336163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7019bf60 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f6e59c700 1 --2- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f7019ba20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f60006af0 tx=0x7f2f60011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f5f7fe700 1 -- 172.21.15.110:0/2539336163 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f60003930 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f5f7fe700 1 -- 172.21.15.110:0/2539336163 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2f60003ad0 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f5f7fe700 1 -- 172.21.15.110:0/2539336163 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f600195e0 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7019c1f0 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.948+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7019eee0 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.949+0000 7f2f5f7fe700 1 -- 172.21.15.110:0/2539336163 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2f60019890 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.949+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f70063890 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.949+0000 7f2f5f7fe700 1 --2- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f2f58039050 0x7f2f5803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.949+0000 7f2f5f7fe700 1 -- 172.21.15.110:0/2539336163 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2f6004d670 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.949+0000 7f2f6dd9b700 1 --2- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f2f58039050 0x7f2f5803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.950+0000 7f2f6dd9b700 1 --2- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f2f58039050 0x7f2f5803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f64006fd0 tx=0x7f2f64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:34.954+0000 7f2f5f7fe700 1 -- 172.21.15.110:0/2539336163 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2f6002d440 con 0x7f2f70107ab0 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:35.153+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi110", "addr": "172.21.15.110", "target": ["mon-mgr", ""]}) -- 0x7f2f70104320 con 0x7f2f58039050 2024-04-24T23:52:39.752 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.165+0000 7f2f5f7fe700 1 -- 172.21.15.110:0/2539336163 <== mgr.14120 v2:172.21.15.110:6800/1818468115 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2f70104320 con 0x7f2f58039050 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f2f58039050 msgr2=0x7f2f5803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 --2- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f2f58039050 0x7f2f5803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f64006fd0 tx=0x7f2f64006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 msgr2=0x7f2f7019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 --2- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f7019ba20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f60006af0 tx=0x7f2f60011b70 comp rx=0 tx=0).stop 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 shutdown_connections 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 --2- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f2f58039050 0x7f2f5803b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 --2- 172.21.15.110:0/2539336163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f70107ab0 0x7f2f7019ba20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 >> 172.21.15.110:0/2539336163 conn(0x7f2f70103090 msgr2=0x7f2f70103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 shutdown_connections 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:39.168+0000 7f2f748dd700 1 -- 172.21.15.110:0/2539336163 wait complete. 2024-04-24T23:52:39.753 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying unmanaged mon service... 2024-04-24T23:52:40.271 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:40 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:40.271 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:40 smithi110 ceph-mon[29187]: Added host smithi110 2024-04-24T23:52:40.271 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:40 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:52:41.665 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:41 smithi110 ceph-mon[29187]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:41.666 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:41 smithi110 ceph-mon[29187]: Saving service mon spec with placement count:5 2024-04-24T23:52:41.666 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:41 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:41.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T23:52:41.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.271+0000 7f98ab20f700 1 Processor -- start 2024-04-24T23:52:41.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.271+0000 7f98ab20f700 1 -- start start 2024-04-24T23:52:41.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.271+0000 7f98ab20f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a4107e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:41.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.271+0000 7f98ab20f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4108420 con 0x7f98a4107a30 2024-04-24T23:52:41.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.272+0000 7f98a8fab700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a4107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:41.908 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.272+0000 7f98a8fab700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a4107e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46882/0 (socket says 172.21.15.110:46882) 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.272+0000 7f98a8fab700 1 -- 172.21.15.110:0/2175395202 learned_addr learned my addr 172.21.15.110:0/2175395202 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.272+0000 7f98a8fab700 1 -- 172.21.15.110:0/2175395202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a4108c30 con 0x7f98a4107a30 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.272+0000 7f98a8fab700 1 --2- 172.21.15.110:0/2175395202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a4107e50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f98980098a0 tx=0x7f9898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6492e56a3e3848 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.272+0000 7f98a37fe700 1 -- 172.21.15.110:0/2175395202 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9898004160 con 0x7f98a4107a30 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.272+0000 7f98a37fe700 1 -- 172.21.15.110:0/2175395202 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9898004300 con 0x7f98a4107a30 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.272+0000 7f98a37fe700 1 -- 172.21.15.110:0/2175395202 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9898011a80 con 0x7f98a4107a30 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 -- 172.21.15.110:0/2175395202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 msgr2=0x7f98a4107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 --2- 172.21.15.110:0/2175395202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a4107e50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f98980098a0 tx=0x7f9898009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 -- 172.21.15.110:0/2175395202 shutdown_connections 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 --2- 172.21.15.110:0/2175395202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a4107e50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 -- 172.21.15.110:0/2175395202 >> 172.21.15.110:0/2175395202 conn(0x7f98a4102ff0 msgr2=0x7f98a4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:41.909 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 -- 172.21.15.110:0/2175395202 shutdown_connections 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 -- 172.21.15.110:0/2175395202 wait complete. 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 Processor -- start 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.273+0000 7f98ab20f700 1 -- start start 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98ab20f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98ab20f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410bfe0 con 0x7f98a4107a30 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98a8fab700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98a8fab700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46890/0 (socket says 172.21.15.110:46890) 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98a8fab700 1 -- 172.21.15.110:0/2094027321 learned_addr learned my addr 172.21.15.110:0/2094027321 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98a8fab700 1 -- 172.21.15.110:0/2094027321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a419bee0 con 0x7f98a4107a30 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98a8fab700 1 --2- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a419b9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9898006af0 tx=0x7f9898011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98a1ffb700 1 -- 172.21.15.110:0/2094027321 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98980039e0 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98a1ffb700 1 -- 172.21.15.110:0/2094027321 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9898003b80 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a419c170 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.274+0000 7f98a1ffb700 1 -- 172.21.15.110:0/2094027321 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98980196a0 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.275+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a419ee60 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.275+0000 7f98a1ffb700 1 -- 172.21.15.110:0/2094027321 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f98980199a0 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.275+0000 7f98a1ffb700 1 --2- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f9894039000 0x7f989403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.275+0000 7f98a1ffb700 1 -- 172.21.15.110:0/2094027321 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f989804d740 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.276+0000 7f98a3fff700 1 --2- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f9894039000 0x7f989403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.276+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4063890 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.276+0000 7f98a3fff700 1 --2- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f9894039000 0x7f989403b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9890006fd0 tx=0x7f9890006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.281+0000 7f98a1ffb700 1 -- 172.21.15.110:0/2094027321 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f98980042a0 con 0x7f98a4107a30 2024-04-24T23:52:41.911 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.503+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f98a4104240 con 0x7f9894039000 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.508+0000 7f98a1ffb700 1 -- 172.21.15.110:0/2094027321 <== mgr.14120 v2:172.21.15.110:6800/1818468115 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f98a4104240 con 0x7f9894039000 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f9894039000 msgr2=0x7f989403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 --2- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f9894039000 0x7f989403b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9890006fd0 tx=0x7f9890006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 msgr2=0x7f98a419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 --2- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a419b9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9898006af0 tx=0x7f9898011c40 comp rx=0 tx=0).stop 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 shutdown_connections 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 --2- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f9894039000 0x7f989403b4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 --2- 172.21.15.110:0/2094027321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98a4107a30 0x7f98a419b9a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 >> 172.21.15.110:0/2094027321 conn(0x7f98a4102ff0 msgr2=0x7f98a4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 shutdown_connections 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:40.510+0000 7f98ab20f700 1 -- 172.21.15.110:0/2094027321 wait complete. 2024-04-24T23:52:41.912 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying unmanaged mgr service... 2024-04-24T23:52:42.974 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:42 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:42.975 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:42 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:42.975 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:42 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.380+0000 7fb61802a700 1 Processor -- start 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.380+0000 7fb61802a700 1 -- start start 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.380+0000 7fb61802a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb610108380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.380+0000 7fb61802a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb610108950 con 0x7fb610107f60 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.380+0000 7fb615dc6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb610108380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.380+0000 7fb615dc6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb610108380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46894/0 (socket says 172.21.15.110:46894) 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.380+0000 7fb615dc6700 1 -- 172.21.15.110:0/2591805910 learned_addr learned my addr 172.21.15.110:0/2591805910 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:43.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.381+0000 7fb615dc6700 1 -- 172.21.15.110:0/2591805910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb610108ad0 con 0x7fb610107f60 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.381+0000 7fb615dc6700 1 --2- 172.21.15.110:0/2591805910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb610108380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0098a0 tx=0x7fb60c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1f835ecb5865a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.381+0000 7fb614dc4700 1 -- 172.21.15.110:0/2591805910 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb60c004160 con 0x7fb610107f60 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.381+0000 7fb614dc4700 1 -- 172.21.15.110:0/2591805910 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb60c004300 con 0x7fb610107f60 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.381+0000 7fb614dc4700 1 -- 172.21.15.110:0/2591805910 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb60c011a50 con 0x7fb610107f60 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 -- 172.21.15.110:0/2591805910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 msgr2=0x7fb610108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 --2- 172.21.15.110:0/2591805910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb610108380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0098a0 tx=0x7fb60c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 -- 172.21.15.110:0/2591805910 shutdown_connections 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 --2- 172.21.15.110:0/2591805910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb610108380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 -- 172.21.15.110:0/2591805910 >> 172.21.15.110:0/2591805910 conn(0x7fb610101630 msgr2=0x7fb610103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 -- 172.21.15.110:0/2591805910 shutdown_connections 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 -- 172.21.15.110:0/2591805910 wait complete. 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 Processor -- start 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.382+0000 7fb61802a700 1 -- start start 2024-04-24T23:52:43.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb61802a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb61019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb61802a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb610106920 con 0x7fb610107f60 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb615dc6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb61019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb615dc6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb61019db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46906/0 (socket says 172.21.15.110:46906) 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb615dc6700 1 -- 172.21.15.110:0/3538406054 learned_addr learned my addr 172.21.15.110:0/3538406054 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb615dc6700 1 -- 172.21.15.110:0/3538406054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61019e040 con 0x7fb610107f60 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb615dc6700 1 --2- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb61019db00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb60c011710 tx=0x7fb60c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb606ffd700 1 -- 172.21.15.110:0/3538406054 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb60c0039c0 con 0x7fb610107f60 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.383+0000 7fb606ffd700 1 -- 172.21.15.110:0/3538406054 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb60c003b60 con 0x7fb610107f60 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.384+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61019e330 con 0x7fb610107f60 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.384+0000 7fb606ffd700 1 -- 172.21.15.110:0/3538406054 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb60c01a690 con 0x7fb610107f60 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.384+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6101a0fc0 con 0x7fb610107f60 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.385+0000 7fb606ffd700 1 -- 172.21.15.110:0/3538406054 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb60c01a9c0 con 0x7fb610107f60 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.385+0000 7fb606ffd700 1 --2- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fb5fc039050 0x7fb5fc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:43.210 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.385+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb610063890 con 0x7fb610107f60 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.385+0000 7fb606ffd700 1 -- 172.21.15.110:0/3538406054 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb60c04d880 con 0x7fb610107f60 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.385+0000 7fb6155c5700 1 --2- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fb5fc039050 0x7fb5fc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.385+0000 7fb6155c5700 1 --2- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fb5fc039050 0x7fb5fc03b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb600006fd0 tx=0x7fb600006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.390+0000 7fb606ffd700 1 -- 172.21.15.110:0/3538406054 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb60c017ca0 con 0x7fb610107f60 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.610+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb610062670 con 0x7fb5fc039050 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.614+0000 7fb606ffd700 1 -- 172.21.15.110:0/3538406054 <== mgr.14120 v2:172.21.15.110:6800/1818468115 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb610062670 con 0x7fb5fc039050 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fb5fc039050 msgr2=0x7fb5fc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 --2- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fb5fc039050 0x7fb5fc03b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb600006fd0 tx=0x7fb600006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 msgr2=0x7fb61019db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 --2- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb61019db00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb60c011710 tx=0x7fb60c011ad0 comp rx=0 tx=0).stop 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 shutdown_connections 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 --2- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fb5fc039050 0x7fb5fc03b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:43.211 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 --2- 172.21.15.110:0/3538406054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb610107f60 0x7fb61019db00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:43.212 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 >> 172.21.15.110:0/3538406054 conn(0x7fb610101630 msgr2=0x7fb610194140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:43.212 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 shutdown_connections 2024-04-24T23:52:43.212 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:42.616+0000 7fb61802a700 1 -- 172.21.15.110:0/3538406054 wait complete. 2024-04-24T23:52:43.888 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:43 smithi110 ceph-mon[29187]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:43.888 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:43 smithi110 ceph-mon[29187]: Saving service mgr spec with placement count:2 2024-04-24T23:52:44.874 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.648+0000 7f083c9fc700 1 Processor -- start 2024-04-24T23:52:44.875 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.648+0000 7f083c9fc700 1 -- start start 2024-04-24T23:52:44.875 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.648+0000 7f083c9fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f0838107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:44.875 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.648+0000 7f083c9fc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0838108460 con 0x7f0838107a70 2024-04-24T23:52:44.875 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.649+0000 7f083659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f0838107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:44.875 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.649+0000 7f083659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f0838107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46916/0 (socket says 172.21.15.110:46916) 2024-04-24T23:52:44.875 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.649+0000 7f083659c700 1 -- 172.21.15.110:0/3607348389 learned_addr learned my addr 172.21.15.110:0/3607348389 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:44.875 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.649+0000 7f083659c700 1 -- 172.21.15.110:0/3607348389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0838108c20 con 0x7f0838107a70 2024-04-24T23:52:44.875 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.649+0000 7f083659c700 1 --2- 172.21.15.110:0/3607348389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f0838107e90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08280098a0 tx=0x7f0828009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d44eb515a5b449 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.649+0000 7f0835d9b700 1 -- 172.21.15.110:0/3607348389 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0828004160 con 0x7f0838107a70 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.649+0000 7f0835d9b700 1 -- 172.21.15.110:0/3607348389 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0828004300 con 0x7f0838107a70 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.649+0000 7f0835d9b700 1 -- 172.21.15.110:0/3607348389 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0828011a50 con 0x7f0838107a70 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.650+0000 7f083c9fc700 1 -- 172.21.15.110:0/3607348389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 msgr2=0x7f0838107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.650+0000 7f083c9fc700 1 --2- 172.21.15.110:0/3607348389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f0838107e90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08280098a0 tx=0x7f0828009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.650+0000 7f083c9fc700 1 -- 172.21.15.110:0/3607348389 shutdown_connections 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.650+0000 7f083c9fc700 1 --2- 172.21.15.110:0/3607348389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f0838107e90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.650+0000 7f083c9fc700 1 -- 172.21.15.110:0/3607348389 >> 172.21.15.110:0/3607348389 conn(0x7f0838103050 msgr2=0x7f0838105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.651+0000 7f083c9fc700 1 -- 172.21.15.110:0/3607348389 shutdown_connections 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.651+0000 7f083c9fc700 1 -- 172.21.15.110:0/3607348389 wait complete. 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.651+0000 7f083c9fc700 1 Processor -- start 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.651+0000 7f083c9fc700 1 -- start start 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.651+0000 7f083c9fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f083807db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:44.876 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f083c9fc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f083810bfd0 con 0x7f0838107a70 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f083659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f083807db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f083659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f083807db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46928/0 (socket says 172.21.15.110:46928) 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f083659c700 1 -- 172.21.15.110:0/2993948145 learned_addr learned my addr 172.21.15.110:0/2993948145 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f083659c700 1 -- 172.21.15.110:0/2993948145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083807e050 con 0x7f0838107a70 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f083659c700 1 --2- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f083807db10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0828006b20 tx=0x7f0828011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f082f7fe700 1 -- 172.21.15.110:0/2993948145 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08280039e0 con 0x7f0838107a70 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f082f7fe700 1 -- 172.21.15.110:0/2993948145 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f082801a8e0 con 0x7f0838107a70 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f082f7fe700 1 -- 172.21.15.110:0/2993948145 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08280247e0 con 0x7f0838107a70 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083807c240 con 0x7f0838107a70 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.652+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083807c780 con 0x7f0838107a70 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.653+0000 7f082f7fe700 1 -- 172.21.15.110:0/2993948145 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0828022020 con 0x7f0838107a70 2024-04-24T23:52:44.877 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.653+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0838063890 con 0x7f0838107a70 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.654+0000 7f082f7fe700 1 --2- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f08240390a0 0x7f082403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.654+0000 7f082f7fe700 1 -- 172.21.15.110:0/2993948145 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f082804d660 con 0x7f0838107a70 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.654+0000 7f082ffff700 1 --2- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f08240390a0 0x7f082403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.654+0000 7f082ffff700 1 --2- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f08240390a0 0x7f082403b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0820006fd0 tx=0x7f0820006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.659+0000 7f082f7fe700 1 -- 172.21.15.110:0/2993948145 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f082800b3e0 con 0x7f0838107a70 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.872+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f083807d020 con 0x7f0838107a70 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.874+0000 7f082f7fe700 1 -- 172.21.15.110:0/2993948145 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0828021d80 con 0x7f0838107a70 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f08240390a0 msgr2=0x7f082403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 --2- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f08240390a0 0x7f082403b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0820006fd0 tx=0x7f0820006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 msgr2=0x7f083807db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 --2- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f083807db10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0828006b20 tx=0x7f0828011ad0 comp rx=0 tx=0).stop 2024-04-24T23:52:44.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 shutdown_connections 2024-04-24T23:52:44.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 --2- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f08240390a0 0x7f082403b560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:44.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 --2- 172.21.15.110:0/2993948145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0838107a70 0x7f083807db10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:44.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 >> 172.21.15.110:0/2993948145 conn(0x7f0838103050 msgr2=0x7f0838103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:44.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 shutdown_connections 2024-04-24T23:52:44.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:43.877+0000 7f083c9fc700 1 -- 172.21.15.110:0/2993948145 wait complete. 2024-04-24T23:52:45.140 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:44 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2993948145' entity='client.admin' 2024-04-24T23:52:45.140 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:44 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:46.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.385+0000 7f4a11d3c700 1 Processor -- start 2024-04-24T23:52:46.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.385+0000 7f4a11d3c700 1 -- start start 2024-04-24T23:52:46.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.385+0000 7f4a11d3c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c10a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:46.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.385+0000 7f4a11d3c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c07bdf0 con 0x7f4a0c108510 2024-04-24T23:52:46.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.385+0000 7f4a0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c10a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:46.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.385+0000 7f4a0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c10a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46944/0 (socket says 172.21.15.110:46944) 2024-04-24T23:52:46.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.385+0000 7f4a0b7fe700 1 -- 172.21.15.110:0/1999987151 learned_addr learned my addr 172.21.15.110:0/1999987151 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:46.181 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.386+0000 7f4a0b7fe700 1 -- 172.21.15.110:0/1999987151 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c10ae40 con 0x7f4a0c108510 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.386+0000 7f4a0b7fe700 1 --2- 172.21.15.110:0/1999987151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c10a900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0098a0 tx=0x7f49fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7754d712e897ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.386+0000 7f4a0a7fc700 1 -- 172.21.15.110:0/1999987151 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49fc004160 con 0x7f4a0c108510 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.386+0000 7f4a0a7fc700 1 -- 172.21.15.110:0/1999987151 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49fc004300 con 0x7f4a0c108510 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.386+0000 7f4a0a7fc700 1 -- 172.21.15.110:0/1999987151 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49fc011a50 con 0x7f4a0c108510 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.387+0000 7f4a11d3c700 1 -- 172.21.15.110:0/1999987151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 msgr2=0x7f4a0c10a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.387+0000 7f4a11d3c700 1 --2- 172.21.15.110:0/1999987151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c10a900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0098a0 tx=0x7f49fc009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.387+0000 7f4a11d3c700 1 -- 172.21.15.110:0/1999987151 shutdown_connections 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.387+0000 7f4a11d3c700 1 --2- 172.21.15.110:0/1999987151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c10a900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.387+0000 7f4a11d3c700 1 -- 172.21.15.110:0/1999987151 >> 172.21.15.110:0/1999987151 conn(0x7f4a0c101ea0 msgr2=0x7f4a0c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.387+0000 7f4a11d3c700 1 -- 172.21.15.110:0/1999987151 shutdown_connections 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.387+0000 7f4a11d3c700 1 -- 172.21.15.110:0/1999987151 wait complete. 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.388+0000 7f4a11d3c700 1 Processor -- start 2024-04-24T23:52:46.182 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.388+0000 7f4a11d3c700 1 -- start start 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.388+0000 7f4a11d3c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.388+0000 7f4a11d3c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10bdf0 con 0x7f4a0c108510 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.388+0000 7f4a0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.388+0000 7f4a0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c19b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46960/0 (socket says 172.21.15.110:46960) 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.388+0000 7f4a0b7fe700 1 -- 172.21.15.110:0/179636234 learned_addr learned my addr 172.21.15.110:0/179636234 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.388+0000 7f4a0b7fe700 1 -- 172.21.15.110:0/179636234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c19be90 con 0x7f4a0c108510 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.389+0000 7f4a0b7fe700 1 --2- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c19b950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49fc011a00 tx=0x7f49fc011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.389+0000 7f4a08ff9700 1 -- 172.21.15.110:0/179636234 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49fc003af0 con 0x7f4a0c108510 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.389+0000 7f4a08ff9700 1 -- 172.21.15.110:0/179636234 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49fc01a8e0 con 0x7f4a0c108510 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.389+0000 7f4a08ff9700 1 -- 172.21.15.110:0/179636234 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49fc024840 con 0x7f4a0c108510 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.389+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c19c120 con 0x7f4a0c108510 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.389+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c106300 con 0x7f4a0c108510 2024-04-24T23:52:46.183 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.390+0000 7f4a08ff9700 1 -- 172.21.15.110:0/179636234 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f49fc022020 con 0x7f4a0c108510 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.390+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c063890 con 0x7f4a0c108510 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.390+0000 7f4a08ff9700 1 --2- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f49f403d460 0x7f49f403f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.390+0000 7f4a08ff9700 1 -- 172.21.15.110:0/179636234 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f49fc04d3b0 con 0x7f4a0c108510 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.391+0000 7f4a0affd700 1 --2- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f49f403d460 0x7f49f403f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.391+0000 7f4a0affd700 1 --2- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f49f403d460 0x7f49f403f920 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a00006fd0 tx=0x7f4a00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.396+0000 7f4a08ff9700 1 -- 172.21.15.110:0/179636234 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f49fc021d80 con 0x7f4a0c108510 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.620+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4a0c106920 con 0x7f4a0c108510 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.623+0000 7f4a08ff9700 1 -- 172.21.15.110:0/179636234 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f49fc02c350 con 0x7f4a0c108510 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.625+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f49f403d460 msgr2=0x7f49f403f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.625+0000 7f4a11d3c700 1 --2- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f49f403d460 0x7f49f403f920 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a00006fd0 tx=0x7f4a00006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.625+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 msgr2=0x7f4a0c19b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.625+0000 7f4a11d3c700 1 --2- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c19b950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49fc011a00 tx=0x7f49fc011ae0 comp rx=0 tx=0).stop 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.625+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 shutdown_connections 2024-04-24T23:52:46.184 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.625+0000 7f4a11d3c700 1 --2- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f49f403d460 0x7f49f403f920 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:46.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.625+0000 7f4a11d3c700 1 --2- 172.21.15.110:0/179636234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a0c108510 0x7f4a0c19b950 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:46.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.625+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 >> 172.21.15.110:0/179636234 conn(0x7f4a0c101ea0 msgr2=0x7f4a0c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:46.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.626+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 shutdown_connections 2024-04-24T23:52:46.185 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:45.626+0000 7f4a11d3c700 1 -- 172.21.15.110:0/179636234 wait complete. 2024-04-24T23:52:46.185 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling the dashboard module... 2024-04-24T23:52:46.817 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:46 smithi110 ceph-mon[29187]: from='mgr.14120 172.21.15.110:0/3919098438' entity='mgr.x' 2024-04-24T23:52:46.817 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:46 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/179636234' entity='client.admin' 2024-04-24T23:52:47.868 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:47 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2567665863' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T23:52:47.868 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:47 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: ignoring --setuser ceph since I am not root 2024-04-24T23:52:47.868 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:47 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: ignoring --setgroup ceph since I am not root 2024-04-24T23:52:47.868 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:47 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:47.746+0000 7f51e4c69200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:52:48.237 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:47 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:47.931+0000 7f51e4c69200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:52:48.237 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:48 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:48.033+0000 7f51e4c69200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:52:48.546 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:48 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:48.342+0000 7f51e4c69200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:52:48.547 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:48 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:48.426+0000 7f51e4c69200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:52:48.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.660+0000 7f0bff621700 1 Processor -- start 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.660+0000 7f0bff621700 1 -- start start 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.661+0000 7f0bff621700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.661+0000 7f0bff621700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf807c870 con 0x7f0bf807ddf0 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.661+0000 7f0bfd3bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.661+0000 7f0bfd3bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33560/0 (socket says 172.21.15.110:33560) 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.661+0000 7f0bfd3bd700 1 -- 172.21.15.110:0/1078693519 learned_addr learned my addr 172.21.15.110:0/1078693519 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.661+0000 7f0bfd3bd700 1 -- 172.21.15.110:0/1078693519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf807c9f0 con 0x7f0bf807ddf0 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.661+0000 7f0bfd3bd700 1 --2- 172.21.15.110:0/1078693519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf807c2a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40098a0 tx=0x7f0bf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea1a6fb9330adb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.662+0000 7f0beffff700 1 -- 172.21.15.110:0/1078693519 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bf4004160 con 0x7f0bf807ddf0 2024-04-24T23:52:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.662+0000 7f0beffff700 1 -- 172.21.15.110:0/1078693519 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bf4004300 con 0x7f0bf807ddf0 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.662+0000 7f0beffff700 1 -- 172.21.15.110:0/1078693519 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bf4011a50 con 0x7f0bf807ddf0 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.662+0000 7f0bff621700 1 -- 172.21.15.110:0/1078693519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 msgr2=0x7f0bf807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.662+0000 7f0bff621700 1 --2- 172.21.15.110:0/1078693519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf807c2a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40098a0 tx=0x7f0bf4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.662+0000 7f0bff621700 1 -- 172.21.15.110:0/1078693519 shutdown_connections 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.662+0000 7f0bff621700 1 --2- 172.21.15.110:0/1078693519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf807c2a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.662+0000 7f0bff621700 1 -- 172.21.15.110:0/1078693519 >> 172.21.15.110:0/1078693519 conn(0x7f0bf8103090 msgr2=0x7f0bf81054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bff621700 1 -- 172.21.15.110:0/1078693519 shutdown_connections 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bff621700 1 -- 172.21.15.110:0/1078693519 wait complete. 2024-04-24T23:52:48.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bff621700 1 Processor -- start 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bff621700 1 -- start start 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bff621700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf810ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bff621700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf810e050 con 0x7f0bf807ddf0 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bfd3bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf810ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bfd3bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf810ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33576/0 (socket says 172.21.15.110:33576) 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.663+0000 7f0bfd3bd700 1 -- 172.21.15.110:0/2567665863 learned_addr learned my addr 172.21.15.110:0/2567665863 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.664+0000 7f0bfd3bd700 1 -- 172.21.15.110:0/2567665863 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf810b280 con 0x7f0bf807ddf0 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.664+0000 7f0bfd3bd700 1 --2- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf810ad40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4011a20 tx=0x7f0bf4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.664+0000 7f0bee7fc700 1 -- 172.21.15.110:0/2567665863 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bf40039c0 con 0x7f0bf807ddf0 2024-04-24T23:52:48.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.664+0000 7f0bee7fc700 1 -- 172.21.15.110:0/2567665863 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bf4003b60 con 0x7f0bf807ddf0 2024-04-24T23:52:48.567 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.664+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf810b510 con 0x7f0bf807ddf0 2024-04-24T23:52:48.567 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.664+0000 7f0bee7fc700 1 -- 172.21.15.110:0/2567665863 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bf401a6b0 con 0x7f0bf807ddf0 2024-04-24T23:52:48.567 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.664+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf8107ad0 con 0x7f0bf807ddf0 2024-04-24T23:52:48.567 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.665+0000 7f0bee7fc700 1 -- 172.21.15.110:0/2567665863 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0bf401a9b0 con 0x7f0bf807ddf0 2024-04-24T23:52:48.567 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.665+0000 7f0bee7fc700 1 --2- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f0be4039000 0x7f0be403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:48.567 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.665+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf810b6e0 con 0x7f0bf807ddf0 2024-04-24T23:52:48.567 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.665+0000 7f0bee7fc700 1 -- 172.21.15.110:0/2567665863 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0bf404d850 con 0x7f0bf807ddf0 2024-04-24T23:52:48.567 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.665+0000 7f0bfcbbc700 1 --2- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f0be4039000 0x7f0be403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.666+0000 7f0bfcbbc700 1 --2- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f0be4039000 0x7f0be403b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006fd0 tx=0x7f0be8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.670+0000 7f0bee7fc700 1 -- 172.21.15.110:0/2567665863 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0bf810b6e0 con 0x7f0bf807ddf0 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:46.933+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f0bf8108150 con 0x7f0bf807ddf0 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.565+0000 7f0bee7fc700 1 -- 172.21.15.110:0/2567665863 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f0bf4024a70 con 0x7f0bf807ddf0 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.566+0000 7f0bee7fc700 1 -- 172.21.15.110:0/2567665863 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f0bf401aca0 con 0x7f0bf807ddf0 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f0be4039000 msgr2=0x7f0be403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 --2- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f0be4039000 0x7f0be403b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006fd0 tx=0x7f0be8006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 msgr2=0x7f0bf810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 --2- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf810ad40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4011a20 tx=0x7f0bf4011b00 comp rx=0 tx=0).stop 2024-04-24T23:52:48.568 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 shutdown_connections 2024-04-24T23:52:48.569 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 --2- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f0be4039000 0x7f0be403b4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:48.569 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 --2- 172.21.15.110:0/2567665863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0bf807ddf0 0x7f0bf810ad40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:48.569 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 >> 172.21.15.110:0/2567665863 conn(0x7f0bf8103090 msgr2=0x7f0bf8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:48.569 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 shutdown_connections 2024-04-24T23:52:48.569 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:47.567+0000 7f0bff621700 1 -- 172.21.15.110:0/2567665863 wait complete. 2024-04-24T23:52:48.874 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:48 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2567665863' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T23:52:48.874 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:48 smithi110 ceph-mon[29187]: mgrmap e9: x(active, since 19s) 2024-04-24T23:52:49.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:49 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1253111884' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:52:49.870 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:49 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:49.641+0000 7f51e4c69200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:52:49.870 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:49 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:49.813+0000 7f51e4c69200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:52:49.872 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:52:49.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T23:52:49.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:52:49.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:52:49.878 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.978+0000 7fab4af9a700 1 Processor -- start 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.978+0000 7fab4af9a700 1 -- start start 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab4af9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab441061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab4af9a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab441067a0 con 0x7fab44105db0 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab48d36700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab441061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab48d36700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab441061d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33598/0 (socket says 172.21.15.110:33598) 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab48d36700 1 -- 172.21.15.110:0/1559549981 learned_addr learned my addr 172.21.15.110:0/1559549981 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab48d36700 1 -- 172.21.15.110:0/1559549981 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab44108d50 con 0x7fab44105db0 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab48d36700 1 --2- 172.21.15.110:0/1559549981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab441061d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab380098a0 tx=0x7fab38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c6cc742618262c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab437fe700 1 -- 172.21.15.110:0/1559549981 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab38004160 con 0x7fab44105db0 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab437fe700 1 -- 172.21.15.110:0/1559549981 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab38004300 con 0x7fab44105db0 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.979+0000 7fab437fe700 1 -- 172.21.15.110:0/1559549981 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab38011a50 con 0x7fab44105db0 2024-04-24T23:52:49.879 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.980+0000 7fab4af9a700 1 -- 172.21.15.110:0/1559549981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 msgr2=0x7fab441061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.980+0000 7fab4af9a700 1 --2- 172.21.15.110:0/1559549981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab441061d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab380098a0 tx=0x7fab38009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.980+0000 7fab4af9a700 1 -- 172.21.15.110:0/1559549981 shutdown_connections 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.980+0000 7fab4af9a700 1 --2- 172.21.15.110:0/1559549981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab441061d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.980+0000 7fab4af9a700 1 -- 172.21.15.110:0/1559549981 >> 172.21.15.110:0/1559549981 conn(0x7fab44101630 msgr2=0x7fab44103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.980+0000 7fab4af9a700 1 -- 172.21.15.110:0/1559549981 shutdown_connections 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.980+0000 7fab4af9a700 1 -- 172.21.15.110:0/1559549981 wait complete. 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.981+0000 7fab4af9a700 1 Processor -- start 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.981+0000 7fab4af9a700 1 -- start start 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.981+0000 7fab4af9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab4419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.981+0000 7fab4af9a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab4410c100 con 0x7fab44105db0 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.981+0000 7fab48d36700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab4419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.981+0000 7fab48d36700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab4419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33614/0 (socket says 172.21.15.110:33614) 2024-04-24T23:52:49.880 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.981+0000 7fab48d36700 1 -- 172.21.15.110:0/1253111884 learned_addr learned my addr 172.21.15.110:0/1253111884 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.981+0000 7fab48d36700 1 -- 172.21.15.110:0/1253111884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4419beb0 con 0x7fab44105db0 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.982+0000 7fab48d36700 1 --2- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab4419b970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab38011a20 tx=0x7fab38011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.982+0000 7fab41ffb700 1 -- 172.21.15.110:0/1253111884 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab380039c0 con 0x7fab44105db0 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.982+0000 7fab41ffb700 1 -- 172.21.15.110:0/1253111884 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab38003b60 con 0x7fab44105db0 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.982+0000 7fab41ffb700 1 -- 172.21.15.110:0/1253111884 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab3801a6b0 con 0x7fab44105db0 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.982+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4419c140 con 0x7fab44105db0 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.982+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4419ee30 con 0x7fab44105db0 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.983+0000 7fab41ffb700 1 -- 172.21.15.110:0/1253111884 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fab38024430 con 0x7fab44105db0 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.983+0000 7fab41ffb700 1 --2- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fab340390b0 0x7fab3403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:49.881 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.983+0000 7fab41ffb700 1 -- 172.21.15.110:0/1253111884 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fab3804dac0 con 0x7fab44105db0 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.983+0000 7fab43fff700 1 -- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fab340390b0 msgr2=0x7fab3403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1818468115 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.983+0000 7fab43fff700 1 --2- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fab340390b0 0x7fab3403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.983+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab44063890 con 0x7fab44105db0 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:48.988+0000 7fab41ffb700 1 -- 172.21.15.110:0/1253111884 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab38024a70 con 0x7fab44105db0 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.184+0000 7fab43fff700 1 -- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fab340390b0 msgr2=0x7fab3403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1818468115 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.184+0000 7fab43fff700 1 --2- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fab340390b0 0x7fab3403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.249+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fab4419f000 con 0x7fab44105db0 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.249+0000 7fab41ffb700 1 -- 172.21.15.110:0/1253111884 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fab38024770 con 0x7fab44105db0 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fab340390b0 msgr2=0x7fab3403b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 --2- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fab340390b0 0x7fab3403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 msgr2=0x7fab4419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 --2- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab4419b970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab38011a20 tx=0x7fab38011b00 comp rx=0 tx=0).stop 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 shutdown_connections 2024-04-24T23:52:49.882 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 --2- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7fab340390b0 0x7fab3403b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:49.883 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 --2- 172.21.15.110:0/1253111884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab44105db0 0x7fab4419b970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:49.883 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 >> 172.21.15.110:0/1253111884 conn(0x7fab44101630 msgr2=0x7fab44102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:49.883 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 shutdown_connections 2024-04-24T23:52:49.883 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:49.251+0000 7fab4af9a700 1 -- 172.21.15.110:0/1253111884 wait complete. 2024-04-24T23:52:49.883 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for the mgr to restart... 2024-04-24T23:52:49.883 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr epoch 9... 2024-04-24T23:52:50.157 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:49 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:49.993+0000 7f51e4c69200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:52:50.536 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:50 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:50.285+0000 7f51e4c69200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:52:50.536 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:50 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:50.382+0000 7f51e4c69200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:52:50.998 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:50 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:50.587+0000 7f51e4c69200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:52:50.998 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:50 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:50.780+0000 7f51e4c69200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:52:51.785 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:51 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:51.376+0000 7f51e4c69200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:52:52.098 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:51 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:51.801+0000 7f51e4c69200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:52:52.099 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:51 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:51.898+0000 7f51e4c69200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:52:52.099 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:52 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:52.099+0000 7f51e4c69200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:52:52.962 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:52 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:52.627+0000 7f51e4c69200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:52:52.963 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:52 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:52.717+0000 7f51e4c69200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:52:53.285 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:52 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:52.962+0000 7f51e4c69200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:52:53.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:53 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:53.095+0000 7f51e4c69200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:52:53.286 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:53 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:53.191+0000 7f51e4c69200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:52:53.785 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:53 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:53.307+0000 7f51e4c69200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:52:53.786 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:53 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:53.399+0000 7f51e4c69200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:52:54.476 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:54.208+0000 7f51e4c69200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:52:54.476 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-24T23:52:54.423+0000 7f51e4c69200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: Active manager daemon x restarted 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: Activating manager daemon x 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: mgrmap e10: x(active, starting, since 0.00332673s) 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:52:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:52:54.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: Manager daemon x is now available 2024-04-24T23:52:54.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:52:54.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:52:54.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:52:55.777 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:55 smithi110 ceph-mon[29187]: [24/Apr/2024:23:52:54] ENGINE Bus STARTING 2024-04-24T23:52:55.777 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:55 smithi110 ceph-mon[29187]: mgrmap e11: x(active, since 1.00575s) 2024-04-24T23:52:56.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:52:56.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T23:52:56.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:52:56.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:52:56.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.391+0000 7f0650413700 1 Processor -- start 2024-04-24T23:52:56.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.391+0000 7f0650413700 1 -- start start 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.392+0000 7f0650413700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f0648107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.392+0000 7f0650413700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06481084c0 con 0x7f0648107ad0 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.392+0000 7f064e1af700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f0648107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.392+0000 7f064e1af700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f0648107ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33616/0 (socket says 172.21.15.110:33616) 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.392+0000 7f064e1af700 1 -- 172.21.15.110:0/3587557304 learned_addr learned my addr 172.21.15.110:0/3587557304 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.392+0000 7f064e1af700 1 -- 172.21.15.110:0/3587557304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0648108c80 con 0x7f0648107ad0 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.392+0000 7f064e1af700 1 --2- 172.21.15.110:0/3587557304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f0648107ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06440098a0 tx=0x7f0644009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4369ea6a2f934c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f064d1ad700 1 -- 172.21.15.110:0/3587557304 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0644004160 con 0x7f0648107ad0 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f064d1ad700 1 -- 172.21.15.110:0/3587557304 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0644004300 con 0x7f0648107ad0 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f064d1ad700 1 -- 172.21.15.110:0/3587557304 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0644011a50 con 0x7f0648107ad0 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f0650413700 1 -- 172.21.15.110:0/3587557304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 msgr2=0x7f0648107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f0650413700 1 --2- 172.21.15.110:0/3587557304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f0648107ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06440098a0 tx=0x7f0644009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f0650413700 1 -- 172.21.15.110:0/3587557304 shutdown_connections 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f0650413700 1 --2- 172.21.15.110:0/3587557304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f0648107ef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f0650413700 1 -- 172.21.15.110:0/3587557304 >> 172.21.15.110:0/3587557304 conn(0x7f0648103090 msgr2=0x7f06481054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f0650413700 1 -- 172.21.15.110:0/3587557304 shutdown_connections 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.393+0000 7f0650413700 1 -- 172.21.15.110:0/3587557304 wait complete. 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.394+0000 7f0650413700 1 Processor -- start 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.394+0000 7f0650413700 1 -- start start 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.394+0000 7f0650413700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f064819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.394+0000 7f0650413700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f064810c030 con 0x7f0648107ad0 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.394+0000 7f064e1af700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f064819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.394+0000 7f064e1af700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f064819b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33630/0 (socket says 172.21.15.110:33630) 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.394+0000 7f064e1af700 1 -- 172.21.15.110:0/1153675879 learned_addr learned my addr 172.21.15.110:0/1153675879 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:56.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.395+0000 7f064e1af700 1 -- 172.21.15.110:0/1153675879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064819bee0 con 0x7f0648107ad0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.395+0000 7f064e1af700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f064819b9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0644011710 tx=0x7f0644011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.395+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06440039c0 con 0x7f0648107ad0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.395+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0644003b60 con 0x7f0648107ad0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.395+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064819c170 con 0x7f0648107ad0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.395+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064819ee60 con 0x7f0648107ad0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.395+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f064401a690 con 0x7f0648107ad0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.396+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f064401aa00 con 0x7f0648107ad0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.396+0000 7f063f7fe700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 0x7f063403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.396+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 --> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f062c000d40 con 0x7f06340390b0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.396+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f064404e4b0 con 0x7f0648107ad0 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.396+0000 7f064d9ae700 1 -- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 msgr2=0x7f063403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1818468115 2024-04-24T23:52:56.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.396+0000 7f064d9ae700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 0x7f063403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.597+0000 7f064d9ae700 1 -- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 msgr2=0x7f063403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1818468115 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.597+0000 7f064d9ae700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 0x7f063403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.998+0000 7f064d9ae700 1 -- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 msgr2=0x7f063403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1818468115 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:50.998+0000 7f064d9ae700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 0x7f063403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:51.799+0000 7f064d9ae700 1 -- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 msgr2=0x7f063403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1818468115 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:51.799+0000 7f064d9ae700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 0x7f063403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:53.400+0000 7f064d9ae700 1 -- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 msgr2=0x7f063403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1818468115 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:53.400+0000 7f064d9ae700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 0x7f063403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:54.427+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f064402d910 con 0x7f0648107ad0 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:54.427+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 msgr2=0x7f063403b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:54.427+0000 7f063f7fe700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 0x7f063403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:56.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.428+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f064404f670 con 0x7f0648107ad0 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.429+0000 7f063f7fe700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f063403c980 0x7f063403ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.429+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0644001d30 con 0x7f063403c980 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.429+0000 7f064d9ae700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f063403c980 0x7f063403ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.430+0000 7f064d9ae700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f063403c980 0x7f063403ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0638003a10 tx=0x7f0638009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.430+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0644001d30 con 0x7f063403c980 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f062c002840 con 0x7f063403c980 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f063f7fe700 1 -- 172.21.15.110:0/1153675879 <== mgr.14152 v2:172.21.15.110:6800/546485163 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f062c002840 con 0x7f063403c980 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f063403c980 msgr2=0x7f063403ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f063403c980 0x7f063403ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0638003a10 tx=0x7f0638009270 comp rx=0 tx=0).stop 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 msgr2=0x7f064819b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f064819b9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0644011710 tx=0x7f0644011ad0 comp rx=0 tx=0).stop 2024-04-24T23:52:56.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 shutdown_connections 2024-04-24T23:52:56.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f063403c980 0x7f063403ed70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:56.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0648107ad0 0x7f064819b9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:56.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 --2- 172.21.15.110:0/1153675879 >> [v2:172.21.15.110:6800/1818468115,v1:172.21.15.110:6801/1818468115] conn(0x7f06340390b0 0x7f063403b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:56.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 >> 172.21.15.110:0/1153675879 conn(0x7f0648103090 msgr2=0x7f0648103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:56.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 shutdown_connections 2024-04-24T23:52:56.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:55.437+0000 7f0650413700 1 -- 172.21.15.110:0/1153675879 wait complete. 2024-04-24T23:52:56.372 INFO:teuthology.orchestra.run.smithi110.stdout:mgr epoch 9 is available 2024-04-24T23:52:56.372 INFO:teuthology.orchestra.run.smithi110.stdout:Generating a dashboard self-signed certificate... 2024-04-24T23:52:56.734 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:56 smithi110 ceph-mon[29187]: [24/Apr/2024:23:52:54] ENGINE Serving on https://172.21.15.110:7150 2024-04-24T23:52:56.734 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:56 smithi110 ceph-mon[29187]: [24/Apr/2024:23:52:54] ENGINE Serving on http://172.21.15.110:8765 2024-04-24T23:52:56.735 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:56 smithi110 ceph-mon[29187]: [24/Apr/2024:23:52:54] ENGINE Bus STARTED 2024-04-24T23:52:56.735 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:56 smithi110 ceph-mon[29187]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:52:56.735 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:56 smithi110 ceph-mon[29187]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:52:56.735 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:52:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T23:52:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.837+0000 7f59b58d6700 1 Processor -- start 2024-04-24T23:52:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.837+0000 7f59b58d6700 1 -- start start 2024-04-24T23:52:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.837+0000 7f59b58d6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b010a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.837+0000 7f59b58d6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f59b007be10 con 0x7f59b01084e0 2024-04-24T23:52:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.837+0000 7f59aeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b010a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.837+0000 7f59aeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b010a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38196/0 (socket says 172.21.15.110:38196) 2024-04-24T23:52:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.837+0000 7f59aeffd700 1 -- 172.21.15.110:0/1960489276 learned_addr learned my addr 172.21.15.110:0/1960489276 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.838+0000 7f59aeffd700 1 -- 172.21.15.110:0/1960489276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b010ae10 con 0x7f59b01084e0 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.838+0000 7f59aeffd700 1 --2- 172.21.15.110:0/1960489276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b010a8d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59a40098a0 tx=0x7f59a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c59e5ec92bc61e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.838+0000 7f59adffb700 1 -- 172.21.15.110:0/1960489276 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a4004160 con 0x7f59b01084e0 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.838+0000 7f59adffb700 1 -- 172.21.15.110:0/1960489276 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59a4004300 con 0x7f59b01084e0 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.838+0000 7f59adffb700 1 -- 172.21.15.110:0/1960489276 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a4011a50 con 0x7f59b01084e0 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.839+0000 7f59b58d6700 1 -- 172.21.15.110:0/1960489276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 msgr2=0x7f59b010a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.839+0000 7f59b58d6700 1 --2- 172.21.15.110:0/1960489276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b010a8d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59a40098a0 tx=0x7f59a4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.839+0000 7f59b58d6700 1 -- 172.21.15.110:0/1960489276 shutdown_connections 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.839+0000 7f59b58d6700 1 --2- 172.21.15.110:0/1960489276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b010a8d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.839+0000 7f59b58d6700 1 -- 172.21.15.110:0/1960489276 >> 172.21.15.110:0/1960489276 conn(0x7f59b0101ed0 msgr2=0x7f59b0104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.839+0000 7f59b58d6700 1 -- 172.21.15.110:0/1960489276 shutdown_connections 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.839+0000 7f59b58d6700 1 -- 172.21.15.110:0/1960489276 wait complete. 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.840+0000 7f59b58d6700 1 Processor -- start 2024-04-24T23:52:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.840+0000 7f59b58d6700 1 -- start start 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.840+0000 7f59b58d6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.840+0000 7f59b58d6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f59b010be10 con 0x7f59b01084e0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.840+0000 7f59aeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.840+0000 7f59aeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38200/0 (socket says 172.21.15.110:38200) 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.840+0000 7f59aeffd700 1 -- 172.21.15.110:0/1112945197 learned_addr learned my addr 172.21.15.110:0/1112945197 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.841+0000 7f59aeffd700 1 -- 172.21.15.110:0/1112945197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b019beb0 con 0x7f59b01084e0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.841+0000 7f59aeffd700 1 --2- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b019b970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59a4011a20 tx=0x7f59a4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.841+0000 7f59b48d4700 1 -- 172.21.15.110:0/1112945197 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a40039c0 con 0x7f59b01084e0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.841+0000 7f59b48d4700 1 -- 172.21.15.110:0/1112945197 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59a4003b60 con 0x7f59b01084e0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.841+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b019c140 con 0x7f59b01084e0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.841+0000 7f59b48d4700 1 -- 172.21.15.110:0/1112945197 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a401a6b0 con 0x7f59b01084e0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.841+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b019ee30 con 0x7f59b01084e0 2024-04-24T23:52:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.842+0000 7f59b48d4700 1 -- 172.21.15.110:0/1112945197 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f59a401a960 con 0x7f59b01084e0 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.842+0000 7f59b48d4700 1 --2- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5998038f90 0x7f599803b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.842+0000 7f59b48d4700 1 -- 172.21.15.110:0/1112945197 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f59a404d7d0 con 0x7f59b01084e0 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.842+0000 7f59ae7fc700 1 --2- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5998038f90 0x7f599803b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.843+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b0063890 con 0x7f59b01084e0 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.843+0000 7f59ae7fc700 1 --2- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5998038f90 0x7f599803b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59a0006fd0 tx=0x7f59a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:56.848+0000 7f59b48d4700 1 -- 172.21.15.110:0/1112945197 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59a400ba30 con 0x7f59b01084e0 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.035+0000 7f59b48d4700 1 -- 172.21.15.110:0/1112945197 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f59a401aca0 con 0x7f59b01084e0 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.055+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f59b01030f0 con 0x7f5998038f90 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.218+0000 7f59b48d4700 1 -- 172.21.15.110:0/1112945197 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f59b01030f0 con 0x7f5998038f90 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5998038f90 msgr2=0x7f599803b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 --2- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5998038f90 0x7f599803b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59a0006fd0 tx=0x7f59a0006e40 comp rx=0 tx=0).stop 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 msgr2=0x7f59b019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 --2- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b019b970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59a4011a20 tx=0x7f59a4011b00 comp rx=0 tx=0).stop 2024-04-24T23:52:57.759 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 shutdown_connections 2024-04-24T23:52:57.760 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 --2- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5998038f90 0x7f599803b450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:57.760 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 --2- 172.21.15.110:0/1112945197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59b01084e0 0x7f59b019b970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:52:57.760 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 >> 172.21.15.110:0/1112945197 conn(0x7f59b0101ed0 msgr2=0x7f59b01029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:52:57.760 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 shutdown_connections 2024-04-24T23:52:57.760 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:57.220+0000 7f59b58d6700 1 -- 172.21.15.110:0/1112945197 wait complete. 2024-04-24T23:52:57.760 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial admin user... 2024-04-24T23:52:58.140 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:58 smithi110 ceph-mon[29187]: mgrmap e12: x(active, since 2s) 2024-04-24T23:52:58.140 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:58 smithi110 ceph-mon[29187]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:58.140 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:52:58.140 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:52:58.141 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:52:59.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:59 smithi110 ceph-mon[29187]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:52:59.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:52:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$kJ.Dvr.UDrEWMvUovP/arOL5TkSsbuavbCCHOt5WdXEAH.8w.CCzi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714002778, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.242+0000 7f3b12237700 1 Processor -- start 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.242+0000 7f3b12237700 1 -- start start 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.242+0000 7f3b12237700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.242+0000 7f3b12237700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c07be10 con 0x7f3b0c106330 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.242+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.242+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38202/0 (socket says 172.21.15.110:38202) 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.242+0000 7f3b0b7fe700 1 -- 172.21.15.110:0/3675796529 learned_addr learned my addr 172.21.15.110:0/3675796529 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.243+0000 7f3b0b7fe700 1 -- 172.21.15.110:0/3675796529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c106c90 con 0x7f3b0c106330 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.243+0000 7f3b0b7fe700 1 --2- 172.21.15.110:0/3675796529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c106750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0098a0 tx=0x7f3afc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b02afecceda3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:00.013 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.243+0000 7f3b0a7fc700 1 -- 172.21.15.110:0/3675796529 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3afc004160 con 0x7f3b0c106330 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.243+0000 7f3b0a7fc700 1 -- 172.21.15.110:0/3675796529 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3afc004300 con 0x7f3b0c106330 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.243+0000 7f3b0a7fc700 1 -- 172.21.15.110:0/3675796529 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3afc011a50 con 0x7f3b0c106330 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.243+0000 7f3b12237700 1 -- 172.21.15.110:0/3675796529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 msgr2=0x7f3b0c106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.243+0000 7f3b12237700 1 --2- 172.21.15.110:0/3675796529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c106750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0098a0 tx=0x7f3afc009bb0 comp rx=0 tx=0).stop 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.244+0000 7f3b12237700 1 -- 172.21.15.110:0/3675796529 shutdown_connections 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.244+0000 7f3b12237700 1 --2- 172.21.15.110:0/3675796529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c106750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.244+0000 7f3b12237700 1 -- 172.21.15.110:0/3675796529 >> 172.21.15.110:0/3675796529 conn(0x7f3b0c101f70 msgr2=0x7f3b0c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.244+0000 7f3b12237700 1 -- 172.21.15.110:0/3675796529 shutdown_connections 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.244+0000 7f3b12237700 1 -- 172.21.15.110:0/3675796529 wait complete. 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.244+0000 7f3b12237700 1 Processor -- start 2024-04-24T23:53:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.245+0000 7f3b12237700 1 -- start start 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.245+0000 7f3b12237700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.245+0000 7f3b12237700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c107d00 con 0x7f3b0c106330 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.245+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.245+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c19db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38218/0 (socket says 172.21.15.110:38218) 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.245+0000 7f3b0b7fe700 1 -- 172.21.15.110:0/610636913 learned_addr learned my addr 172.21.15.110:0/610636913 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.245+0000 7f3b0b7fe700 1 -- 172.21.15.110:0/610636913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c19e0d0 con 0x7f3b0c106330 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.246+0000 7f3b0b7fe700 1 --2- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c19db90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3afc011a00 tx=0x7f3afc011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.246+0000 7f3b08ff9700 1 -- 172.21.15.110:0/610636913 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3afc003af0 con 0x7f3b0c106330 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.246+0000 7f3b08ff9700 1 -- 172.21.15.110:0/610636913 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3afc01a8e0 con 0x7f3b0c106330 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.246+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c19e360 con 0x7f3b0c106330 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.246+0000 7f3b08ff9700 1 -- 172.21.15.110:0/610636913 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3afc024840 con 0x7f3b0c106330 2024-04-24T23:53:00.018 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.246+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1a1050 con 0x7f3b0c106330 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.247+0000 7f3b08ff9700 1 -- 172.21.15.110:0/610636913 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3afc022020 con 0x7f3b0c106330 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.247+0000 7f3b08ff9700 1 --2- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3af40390a0 0x7f3af403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.247+0000 7f3b08ff9700 1 -- 172.21.15.110:0/610636913 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3afc04e310 con 0x7f3b0c106330 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.247+0000 7f3b0affd700 1 --2- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3af40390a0 0x7f3af403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.247+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0c063890 con 0x7f3b0c106330 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.248+0000 7f3b0affd700 1 --2- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3af40390a0 0x7f3af403b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b00006fd0 tx=0x7f3b00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.252+0000 7f3b08ff9700 1 -- 172.21.15.110:0/610636913 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3afc0172b0 con 0x7f3b0c106330 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.474+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3b0c02dc40 con 0x7f3af40390a0 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.698+0000 7f3b08ff9700 1 -- 172.21.15.110:0/610636913 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3b0c02dc40 con 0x7f3af40390a0 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3af40390a0 msgr2=0x7f3af403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 --2- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3af40390a0 0x7f3af403b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b00006fd0 tx=0x7f3b00006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 msgr2=0x7f3b0c19db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 --2- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c19db90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3afc011a00 tx=0x7f3afc011ae0 comp rx=0 tx=0).stop 2024-04-24T23:53:00.019 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 shutdown_connections 2024-04-24T23:53:00.020 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 --2- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3af40390a0 0x7f3af403b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:00.020 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 --2- 172.21.15.110:0/610636913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b0c106330 0x7f3b0c19db90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:00.020 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 >> 172.21.15.110:0/610636913 conn(0x7f3b0c101f70 msgr2=0x7f3b0c1947f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:00.020 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 shutdown_connections 2024-04-24T23:53:00.020 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:52:58.700+0000 7f3b12237700 1 -- 172.21.15.110:0/610636913 wait complete. 2024-04-24T23:53:00.020 INFO:teuthology.orchestra.run.smithi110.stdout:Fetching dashboard port number... 2024-04-24T23:53:01.200 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.470+0000 7f721c683700 1 Processor -- start 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.470+0000 7f721c683700 1 -- start start 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.470+0000 7f721c683700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f7214107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.470+0000 7f721c683700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f72141084a0 con 0x7f7214107ab0 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.470+0000 7f721a41f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f7214107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.470+0000 7f721a41f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f7214107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38220/0 (socket says 172.21.15.110:38220) 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.470+0000 7f721a41f700 1 -- 172.21.15.110:0/3854720277 learned_addr learned my addr 172.21.15.110:0/3854720277 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.471+0000 7f721a41f700 1 -- 172.21.15.110:0/3854720277 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7214108c60 con 0x7f7214107ab0 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.471+0000 7f721a41f700 1 --2- 172.21.15.110:0/3854720277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f7214107ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f720c0098a0 tx=0x7f720c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db41c244876ea6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.471+0000 7f721941d700 1 -- 172.21.15.110:0/3854720277 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f720c004160 con 0x7f7214107ab0 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.471+0000 7f721941d700 1 -- 172.21.15.110:0/3854720277 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f720c004300 con 0x7f7214107ab0 2024-04-24T23:53:01.206 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.471+0000 7f721941d700 1 -- 172.21.15.110:0/3854720277 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f720c011a50 con 0x7f7214107ab0 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 -- 172.21.15.110:0/3854720277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 msgr2=0x7f7214107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 --2- 172.21.15.110:0/3854720277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f7214107ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f720c0098a0 tx=0x7f720c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 -- 172.21.15.110:0/3854720277 shutdown_connections 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 --2- 172.21.15.110:0/3854720277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f7214107ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 -- 172.21.15.110:0/3854720277 >> 172.21.15.110:0/3854720277 conn(0x7f7214103090 msgr2=0x7f72141054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 -- 172.21.15.110:0/3854720277 shutdown_connections 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 -- 172.21.15.110:0/3854720277 wait complete. 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 Processor -- start 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 -- start start 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f721419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721c683700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f721410c010 con 0x7f7214107ab0 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.472+0000 7f721a41f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f721419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f721a41f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f721419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38232/0 (socket says 172.21.15.110:38232) 2024-04-24T23:53:01.207 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f721a41f700 1 -- 172.21.15.110:0/3976906597 learned_addr learned my addr 172.21.15.110:0/3976906597 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f721a41f700 1 -- 172.21.15.110:0/3976906597 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721419bef0 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f721a41f700 1 --2- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f721419b9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f720c011710 tx=0x7f720c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f720b7fe700 1 -- 172.21.15.110:0/3976906597 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f720c0039c0 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f720b7fe700 1 -- 172.21.15.110:0/3976906597 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f720c003b60 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721419c180 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f720b7fe700 1 -- 172.21.15.110:0/3976906597 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f720c01a690 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.473+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721419ee70 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.474+0000 7f720b7fe700 1 -- 172.21.15.110:0/3976906597 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f720c01a990 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.474+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7214063890 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.474+0000 7f720b7fe700 1 --2- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7200039050 0x7f720003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.474+0000 7f720b7fe700 1 -- 172.21.15.110:0/3976906597 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f720c04d9c0 con 0x7f7214107ab0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.474+0000 7f7219c1e700 1 --2- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7200039050 0x7f720003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.474+0000 7f7219c1e700 1 --2- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7200039050 0x7f720003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7204006fd0 tx=0x7f7204006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.479+0000 7f720b7fe700 1 -- 172.21.15.110:0/3976906597 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f720c04fe20 con 0x7f7214107ab0 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.481+0000 7f720b7fe700 1 -- 172.21.15.110:0/3976906597 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f720c02d3f0 con 0x7f7214107ab0 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.679+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f721419f420 con 0x7f7214107ab0 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.679+0000 7f720b7fe700 1 -- 172.21.15.110:0/3976906597 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f720c019c70 con 0x7f7214107ab0 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7200039050 msgr2=0x7f720003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 --2- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7200039050 0x7f720003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7204006fd0 tx=0x7f7204006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 msgr2=0x7f721419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 --2- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f721419b9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f720c011710 tx=0x7f720c011ad0 comp rx=0 tx=0).stop 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 shutdown_connections 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 --2- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7200039050 0x7f720003b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 --2- 172.21.15.110:0/3976906597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7214107ab0 0x7f721419b9b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 >> 172.21.15.110:0/3976906597 conn(0x7f7214103090 msgr2=0x7f7214103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.681+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 shutdown_connections 2024-04-24T23:53:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:00.682+0000 7f721c683700 1 -- 172.21.15.110:0/3976906597 wait complete. 2024-04-24T23:53:01.211 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:53:01.211 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-04-24T23:53:01.218 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:53:01.218 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-24T23:53:01.218 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-04-24T23:53:01.218 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T23:53:01.220 INFO:teuthology.orchestra.run.smithi110.stdout:Ceph Dashboard is now available at: 2024-04-24T23:53:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: URL: https://smithi110:8443/ 2024-04-24T23:53:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: User: admin 2024-04-24T23:53:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: Password: 2k29su99tc 2024-04-24T23:53:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:01.221 INFO:teuthology.orchestra.run.smithi110.stdout:Saving cluster configuration to /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config directory 2024-04-24T23:53:01.221 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling autotune for osd_memory_target 2024-04-24T23:53:01.658 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:01 smithi110 ceph-mon[29187]: mgrmap e13: x(active, since 6s) 2024-04-24T23:53:01.658 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:01 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3976906597' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T23:53:02.461 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.677+0000 7fcae8f49700 1 Processor -- start 2024-04-24T23:53:02.461 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.677+0000 7fcae8f49700 1 -- start start 2024-04-24T23:53:02.461 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.677+0000 7fcae8f49700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae4106db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:02.461 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.677+0000 7fcae8f49700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4107380 con 0x7fcae4106990 2024-04-24T23:53:02.461 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.677+0000 7fcae259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae4106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:02.461 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.677+0000 7fcae259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae4106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38238/0 (socket says 172.21.15.110:38238) 2024-04-24T23:53:02.461 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.677+0000 7fcae259c700 1 -- 172.21.15.110:0/1845901399 learned_addr learned my addr 172.21.15.110:0/1845901399 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.678+0000 7fcae259c700 1 -- 172.21.15.110:0/1845901399 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae4107b90 con 0x7fcae4106990 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.678+0000 7fcae259c700 1 --2- 172.21.15.110:0/1845901399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae4106db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcad400bb40 tx=0x7fcad400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856271d66eb7ffde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.678+0000 7fcae1d9b700 1 -- 172.21.15.110:0/1845901399 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad400dbe0 con 0x7fcae4106990 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.678+0000 7fcae1d9b700 1 -- 172.21.15.110:0/1845901399 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcad40044e0 con 0x7fcae4106990 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.678+0000 7fcae1d9b700 1 -- 172.21.15.110:0/1845901399 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad4010470 con 0x7fcae4106990 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.679+0000 7fcae8f49700 1 -- 172.21.15.110:0/1845901399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 msgr2=0x7fcae4106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.679+0000 7fcae8f49700 1 --2- 172.21.15.110:0/1845901399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae4106db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcad400bb40 tx=0x7fcad400be50 comp rx=0 tx=0).stop 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.679+0000 7fcae8f49700 1 -- 172.21.15.110:0/1845901399 shutdown_connections 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.679+0000 7fcae8f49700 1 --2- 172.21.15.110:0/1845901399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae4106db0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.679+0000 7fcae8f49700 1 -- 172.21.15.110:0/1845901399 >> 172.21.15.110:0/1845901399 conn(0x7fcae4101f70 msgr2=0x7fcae4104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:02.462 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.679+0000 7fcae8f49700 1 -- 172.21.15.110:0/1845901399 shutdown_connections 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.679+0000 7fcae8f49700 1 -- 172.21.15.110:0/1845901399 wait complete. 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.679+0000 7fcae8f49700 1 Processor -- start 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae8f49700 1 -- start start 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae8f49700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae8f49700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcae407af70 con 0x7fcae4106990 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae41997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38244/0 (socket says 172.21.15.110:38244) 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae259c700 1 -- 172.21.15.110:0/2142281919 learned_addr learned my addr 172.21.15.110:0/2142281919 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae259c700 1 -- 172.21.15.110:0/2142281919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae4199cf0 con 0x7fcae4106990 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae259c700 1 --2- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae41997b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcad400d650 tx=0x7fcad4003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcadbfff700 1 -- 172.21.15.110:0/2142281919 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad4027070 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcadbfff700 1 -- 172.21.15.110:0/2142281919 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcad4010c20 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae4199f80 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.680+0000 7fcadbfff700 1 -- 172.21.15.110:0/2142281919 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad4022c30 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.681+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae419a3e0 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.681+0000 7fcadbfff700 1 -- 172.21.15.110:0/2142281919 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fcad402a070 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.681+0000 7fcadbfff700 1 --2- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcad00390f0 0x7fcad003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.681+0000 7fcadbfff700 1 -- 172.21.15.110:0/2142281919 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcad401f070 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.682+0000 7fcad9d9b700 1 --2- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcad00390f0 0x7fcad003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.682+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae4063890 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.682+0000 7fcad9d9b700 1 --2- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcad00390f0 0x7fcad003b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcacc006fd0 tx=0x7fcacc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.687+0000 7fcadbfff700 1 -- 172.21.15.110:0/2142281919 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcad4015070 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.886+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fcae4192d10 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.886+0000 7fcadbfff700 1 -- 172.21.15.110:0/2142281919 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fcad4010dd0 con 0x7fcae4106990 2024-04-24T23:53:02.467 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcad00390f0 msgr2=0x7fcad003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 --2- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcad00390f0 0x7fcad003b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcacc006fd0 tx=0x7fcacc006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 msgr2=0x7fcae41997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 --2- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae41997b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcad400d650 tx=0x7fcad4003fb0 comp rx=0 tx=0).stop 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 shutdown_connections 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 --2- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcad00390f0 0x7fcad003b5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 --2- 172.21.15.110:0/2142281919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcae4106990 0x7fcae41997b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 >> 172.21.15.110:0/2142281919 conn(0x7fcae4101f70 msgr2=0x7fcae418fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 shutdown_connections 2024-04-24T23:53:02.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:01.888+0000 7fcae8f49700 1 -- 172.21.15.110:0/2142281919 wait complete. 2024-04-24T23:53:03.818 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.898+0000 7f1f85214700 1 Processor -- start 2024-04-24T23:53:03.818 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f85214700 1 -- start start 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f85214700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f80107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f85214700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801084a0 con 0x7f1f80107ab0 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f7ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f80107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f7ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f80107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38250/0 (socket says 172.21.15.110:38250) 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f7ed9d700 1 -- 172.21.15.110:0/469307736 learned_addr learned my addr 172.21.15.110:0/469307736 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f7ed9d700 1 -- 172.21.15.110:0/469307736 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f80108c60 con 0x7f1f80107ab0 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f7ed9d700 1 --2- 172.21.15.110:0/469307736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f80107ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f700098a0 tx=0x7f1f70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196844bd335132fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f7dd9b700 1 -- 172.21.15.110:0/469307736 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f70004160 con 0x7f1f80107ab0 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f7dd9b700 1 -- 172.21.15.110:0/469307736 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f70004300 con 0x7f1f80107ab0 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.899+0000 7f1f7dd9b700 1 -- 172.21.15.110:0/469307736 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f70011a50 con 0x7f1f80107ab0 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.900+0000 7f1f85214700 1 -- 172.21.15.110:0/469307736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 msgr2=0x7f1f80107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:03.819 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.900+0000 7f1f85214700 1 --2- 172.21.15.110:0/469307736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f80107ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f700098a0 tx=0x7f1f70009bb0 comp rx=0 tx=0).stop 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.900+0000 7f1f85214700 1 -- 172.21.15.110:0/469307736 shutdown_connections 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.900+0000 7f1f85214700 1 --2- 172.21.15.110:0/469307736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f80107ed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.900+0000 7f1f85214700 1 -- 172.21.15.110:0/469307736 >> 172.21.15.110:0/469307736 conn(0x7f1f80103090 msgr2=0x7f1f801054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.900+0000 7f1f85214700 1 -- 172.21.15.110:0/469307736 shutdown_connections 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.900+0000 7f1f85214700 1 -- 172.21.15.110:0/469307736 wait complete. 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.901+0000 7f1f85214700 1 Processor -- start 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.901+0000 7f1f85214700 1 -- start start 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.901+0000 7f1f85214700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f8019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.901+0000 7f1f85214700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8010c010 con 0x7f1f80107ab0 2024-04-24T23:53:03.820 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.901+0000 7f1f7ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f8019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.901+0000 7f1f7ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f8019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38262/0 (socket says 172.21.15.110:38262) 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.901+0000 7f1f7ed9d700 1 -- 172.21.15.110:0/576499926 learned_addr learned my addr 172.21.15.110:0/576499926 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.902+0000 7f1f7ed9d700 1 -- 172.21.15.110:0/576499926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8019bef0 con 0x7f1f80107ab0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.902+0000 7f1f7ed9d700 1 --2- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f8019b9b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f70011a20 tx=0x7f1f70011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.902+0000 7f1f67fff700 1 -- 172.21.15.110:0/576499926 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f70003af0 con 0x7f1f80107ab0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.902+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8019c180 con 0x7f1f80107ab0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.902+0000 7f1f67fff700 1 -- 172.21.15.110:0/576499926 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f7001a8e0 con 0x7f1f80107ab0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.902+0000 7f1f67fff700 1 -- 172.21.15.110:0/576499926 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f70024860 con 0x7f1f80107ab0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.902+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8019ee70 con 0x7f1f80107ab0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.903+0000 7f1f67fff700 1 -- 172.21.15.110:0/576499926 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1f70022020 con 0x7f1f80107ab0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.903+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f80063890 con 0x7f1f80107ab0 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.903+0000 7f1f67fff700 1 --2- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f68039140 0x7f1f6803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.903+0000 7f1f67fff700 1 -- 172.21.15.110:0/576499926 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1f7004d520 con 0x7f1f80107ab0 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.903+0000 7f1f7e59c700 1 --2- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f68039140 0x7f1f6803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.903+0000 7f1f7e59c700 1 --2- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f68039140 0x7f1f6803b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f74006fd0 tx=0x7f1f74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:02.908+0000 7f1f67fff700 1 -- 172.21.15.110:0/576499926 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f7000b3e0 con 0x7f1f80107ab0 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.215+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1f8019f040 con 0x7f1f80107ab0 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.217+0000 7f1f67fff700 1 -- 172.21.15.110:0/576499926 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f1f700172b0 con 0x7f1f80107ab0 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f68039140 msgr2=0x7f1f6803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 --2- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f68039140 0x7f1f6803b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f74006fd0 tx=0x7f1f74006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 msgr2=0x7f1f8019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:03.822 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 --2- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f8019b9b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f70011a20 tx=0x7f1f70011b00 comp rx=0 tx=0).stop 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 shutdown_connections 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 --2- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f68039140 0x7f1f6803b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 --2- 172.21.15.110:0/576499926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f80107ab0 0x7f1f8019b9b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 >> 172.21.15.110:0/576499926 conn(0x7f1f80103090 msgr2=0x7f1f80103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 shutdown_connections 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-24T23:53:03.220+0000 7f1f85214700 1 -- 172.21.15.110:0/576499926 wait complete. 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:03.823 INFO:teuthology.orchestra.run.smithi110.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: ceph telemetry on 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout:For more information see: 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T23:53:03.824 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:03.825 INFO:teuthology.orchestra.run.smithi110.stdout:Bootstrap complete. 2024-04-24T23:53:03.852 INFO:tasks.cephadm:Fetching config... 2024-04-24T23:53:03.853 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:53:03.853 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T23:53:03.867 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T23:53:03.867 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:53:03.867 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T23:53:03.923 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T23:53:03.923 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:53:03.923 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T23:53:03.987 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T23:53:03.987 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:53:03.987 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T23:53:04.043 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T23:53:04.044 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAk9FA9DPlXj9C2FfL4uZ8rYP8dSuFhY1lSIhKbf87qgN28dpn7yOjGsuGbJGpG4vOvepYAcTvCGLOA5LTPHsDz42fXZZ2RJk9yw+1VdGmSywNbcNRi7drW2S7hvLmqLnboBovq25KNUbHuPf4vdw7QRb4qa54FxVZAboDcj+kY++AYcpNR0Gt0pFZ/MQLQB+Bwyju5MEhR250K3w4fD+8iAP+OyfGV9VnF7PWXrCsobOMe4F6qJbADZO5CbK3t4SDp+p2tUkBo4KBhBFWXIddhi7nTSMiXmn5Y6HUf4LkFH727sLGvdaLEai54dbmcrGn5nwHlFbufaHFTgj0Lknu1KmaK+YHcmrOsiGKi7Yja7+v9TqnM03Ona94oIl/ix/Pqb9dDkEopHR8fZi0xhl6ju/4W3Z5oHc9J8+HD0s7jTI6wCrl54g711g+E43V8RdvOoJxZiFWkpTWCepFwFINTuSZPA87jd+AH9uUmOe/uz2HaH2i8IfCJPNWHM9mUrc= ceph-787fa75a-0295-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:53:04.115 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAk9FA9DPlXj9C2FfL4uZ8rYP8dSuFhY1lSIhKbf87qgN28dpn7yOjGsuGbJGpG4vOvepYAcTvCGLOA5LTPHsDz42fXZZ2RJk9yw+1VdGmSywNbcNRi7drW2S7hvLmqLnboBovq25KNUbHuPf4vdw7QRb4qa54FxVZAboDcj+kY++AYcpNR0Gt0pFZ/MQLQB+Bwyju5MEhR250K3w4fD+8iAP+OyfGV9VnF7PWXrCsobOMe4F6qJbADZO5CbK3t4SDp+p2tUkBo4KBhBFWXIddhi7nTSMiXmn5Y6HUf4LkFH727sLGvdaLEai54dbmcrGn5nwHlFbufaHFTgj0Lknu1KmaK+YHcmrOsiGKi7Yja7+v9TqnM03Ona94oIl/ix/Pqb9dDkEopHR8fZi0xhl6ju/4W3Z5oHc9J8+HD0s7jTI6wCrl54g711g+E43V8RdvOoJxZiFWkpTWCepFwFINTuSZPA87jd+AH9uUmOe/uz2HaH2i8IfCJPNWHM9mUrc= ceph-787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:04.125 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAk9FA9DPlXj9C2FfL4uZ8rYP8dSuFhY1lSIhKbf87qgN28dpn7yOjGsuGbJGpG4vOvepYAcTvCGLOA5LTPHsDz42fXZZ2RJk9yw+1VdGmSywNbcNRi7drW2S7hvLmqLnboBovq25KNUbHuPf4vdw7QRb4qa54FxVZAboDcj+kY++AYcpNR0Gt0pFZ/MQLQB+Bwyju5MEhR250K3w4fD+8iAP+OyfGV9VnF7PWXrCsobOMe4F6qJbADZO5CbK3t4SDp+p2tUkBo4KBhBFWXIddhi7nTSMiXmn5Y6HUf4LkFH727sLGvdaLEai54dbmcrGn5nwHlFbufaHFTgj0Lknu1KmaK+YHcmrOsiGKi7Yja7+v9TqnM03Ona94oIl/ix/Pqb9dDkEopHR8fZi0xhl6ju/4W3Z5oHc9J8+HD0s7jTI6wCrl54g711g+E43V8RdvOoJxZiFWkpTWCepFwFINTuSZPA87jd+AH9uUmOe/uz2HaH2i8IfCJPNWHM9mUrc= ceph-787fa75a-0295-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:53:04.164 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAk9FA9DPlXj9C2FfL4uZ8rYP8dSuFhY1lSIhKbf87qgN28dpn7yOjGsuGbJGpG4vOvepYAcTvCGLOA5LTPHsDz42fXZZ2RJk9yw+1VdGmSywNbcNRi7drW2S7hvLmqLnboBovq25KNUbHuPf4vdw7QRb4qa54FxVZAboDcj+kY++AYcpNR0Gt0pFZ/MQLQB+Bwyju5MEhR250K3w4fD+8iAP+OyfGV9VnF7PWXrCsobOMe4F6qJbADZO5CbK3t4SDp+p2tUkBo4KBhBFWXIddhi7nTSMiXmn5Y6HUf4LkFH727sLGvdaLEai54dbmcrGn5nwHlFbufaHFTgj0Lknu1KmaK+YHcmrOsiGKi7Yja7+v9TqnM03Ona94oIl/ix/Pqb9dDkEopHR8fZi0xhl6ju/4W3Z5oHc9J8+HD0s7jTI6wCrl54g711g+E43V8RdvOoJxZiFWkpTWCepFwFINTuSZPA87jd+AH9uUmOe/uz2HaH2i8IfCJPNWHM9mUrc= ceph-787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:04.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAk9FA9DPlXj9C2FfL4uZ8rYP8dSuFhY1lSIhKbf87qgN28dpn7yOjGsuGbJGpG4vOvepYAcTvCGLOA5LTPHsDz42fXZZ2RJk9yw+1VdGmSywNbcNRi7drW2S7hvLmqLnboBovq25KNUbHuPf4vdw7QRb4qa54FxVZAboDcj+kY++AYcpNR0Gt0pFZ/MQLQB+Bwyju5MEhR250K3w4fD+8iAP+OyfGV9VnF7PWXrCsobOMe4F6qJbADZO5CbK3t4SDp+p2tUkBo4KBhBFWXIddhi7nTSMiXmn5Y6HUf4LkFH727sLGvdaLEai54dbmcrGn5nwHlFbufaHFTgj0Lknu1KmaK+YHcmrOsiGKi7Yja7+v9TqnM03Ona94oIl/ix/Pqb9dDkEopHR8fZi0xhl6ju/4W3Z5oHc9J8+HD0s7jTI6wCrl54g711g+E43V8RdvOoJxZiFWkpTWCepFwFINTuSZPA87jd+AH9uUmOe/uz2HaH2i8IfCJPNWHM9mUrc= ceph-787fa75a-0295-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:53:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAk9FA9DPlXj9C2FfL4uZ8rYP8dSuFhY1lSIhKbf87qgN28dpn7yOjGsuGbJGpG4vOvepYAcTvCGLOA5LTPHsDz42fXZZ2RJk9yw+1VdGmSywNbcNRi7drW2S7hvLmqLnboBovq25KNUbHuPf4vdw7QRb4qa54FxVZAboDcj+kY++AYcpNR0Gt0pFZ/MQLQB+Bwyju5MEhR250K3w4fD+8iAP+OyfGV9VnF7PWXrCsobOMe4F6qJbADZO5CbK3t4SDp+p2tUkBo4KBhBFWXIddhi7nTSMiXmn5Y6HUf4LkFH727sLGvdaLEai54dbmcrGn5nwHlFbufaHFTgj0Lknu1KmaK+YHcmrOsiGKi7Yja7+v9TqnM03Ona94oIl/ix/Pqb9dDkEopHR8fZi0xhl6ju/4W3Z5oHc9J8+HD0s7jTI6wCrl54g711g+E43V8RdvOoJxZiFWkpTWCepFwFINTuSZPA87jd+AH9uUmOe/uz2HaH2i8IfCJPNWHM9mUrc= ceph-787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:04.226 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T23:53:04.460 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:53:04.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:04 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/576499926' entity='client.admin' 2024-04-24T23:53:07.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.176+0000 7f534b872700 1 -- 172.21.15.110:0/4098008283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 msgr2=0x7f53441025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:07.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.176+0000 7f534b872700 1 --2- 172.21.15.110:0/4098008283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 0x7f53441025a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5340009910 tx=0x7f5340009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:07.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.176+0000 7f534b872700 1 -- 172.21.15.110:0/4098008283 shutdown_connections 2024-04-24T23:53:07.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.176+0000 7f534b872700 1 --2- 172.21.15.110:0/4098008283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 0x7f53441025a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:07.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.176+0000 7f534b872700 1 -- 172.21.15.110:0/4098008283 >> 172.21.15.110:0/4098008283 conn(0x7f53440fb910 msgr2=0x7f53440fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:07.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.176+0000 7f534b872700 1 -- 172.21.15.110:0/4098008283 shutdown_connections 2024-04-24T23:53:07.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.176+0000 7f534b872700 1 -- 172.21.15.110:0/4098008283 wait complete. 2024-04-24T23:53:07.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.177+0000 7f534b872700 1 Processor -- start 2024-04-24T23:53:07.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.177+0000 7f534b872700 1 -- start start 2024-04-24T23:53:07.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.177+0000 7f534b872700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 0x7f53441972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:07.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.177+0000 7f534b872700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5344106ac0 con 0x7f5344102180 2024-04-24T23:53:07.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.177+0000 7f534960e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 0x7f53441972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:07.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.177+0000 7f534960e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 0x7f53441972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35796/0 (socket says 172.21.15.110:35796) 2024-04-24T23:53:07.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.177+0000 7f534960e700 1 -- 172.21.15.110:0/1551495439 learned_addr learned my addr 172.21.15.110:0/1551495439 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:07.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.178+0000 7f534960e700 1 -- 172.21.15.110:0/1551495439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5344197810 con 0x7f5344102180 2024-04-24T23:53:07.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.178+0000 7f534960e700 1 --2- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 0x7f53441972d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5340005800 tx=0x7f5340009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:07.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.178+0000 7f533a7fc700 1 -- 172.21.15.110:0/1551495439 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5340027070 con 0x7f5344102180 2024-04-24T23:53:07.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.178+0000 7f533a7fc700 1 -- 172.21.15.110:0/1551495439 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f534000be10 con 0x7f5344102180 2024-04-24T23:53:07.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.178+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5344197aa0 con 0x7f5344102180 2024-04-24T23:53:07.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.178+0000 7f533a7fc700 1 -- 172.21.15.110:0/1551495439 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f534000f460 con 0x7f5344102180 2024-04-24T23:53:07.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.178+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534419a790 con 0x7f5344102180 2024-04-24T23:53:07.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.179+0000 7f533a7fc700 1 -- 172.21.15.110:0/1551495439 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f534002a070 con 0x7f5344102180 2024-04-24T23:53:07.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.179+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5344107240 con 0x7f5344102180 2024-04-24T23:53:07.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.179+0000 7f533a7fc700 1 --2- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5330039150 0x7f533003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:07.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.179+0000 7f533a7fc700 1 -- 172.21.15.110:0/1551495439 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f534001e030 con 0x7f5344102180 2024-04-24T23:53:07.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.180+0000 7f5348e0d700 1 --2- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5330039150 0x7f533003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:07.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.180+0000 7f5348e0d700 1 --2- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5330039150 0x7f533003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5334006fd0 tx=0x7f5334006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:07.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.187+0000 7f533a7fc700 1 -- 172.21.15.110:0/1551495439 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5340013070 con 0x7f5344102180 2024-04-24T23:53:07.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.414+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f534402dc40 con 0x7f5344102180 2024-04-24T23:53:07.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.416+0000 7f533a7fc700 1 -- 172.21.15.110:0/1551495439 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5340031020 con 0x7f5344102180 2024-04-24T23:53:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.418+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5330039150 msgr2=0x7f533003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:07.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.418+0000 7f534b872700 1 --2- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5330039150 0x7f533003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5334006fd0 tx=0x7f5334006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.418+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 msgr2=0x7f53441972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.418+0000 7f534b872700 1 --2- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 0x7f53441972d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5340005800 tx=0x7f5340009f60 comp rx=0 tx=0).stop 2024-04-24T23:53:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.418+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 shutdown_connections 2024-04-24T23:53:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.418+0000 7f534b872700 1 --2- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5330039150 0x7f533003b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.418+0000 7f534b872700 1 --2- 172.21.15.110:0/1551495439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5344102180 0x7f53441972d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.418+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 >> 172.21.15.110:0/1551495439 conn(0x7f53440fb910 msgr2=0x7f5344100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.419+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 shutdown_connections 2024-04-24T23:53:07.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:07.419+0000 7f534b872700 1 -- 172.21.15.110:0/1551495439 wait complete. 2024-04-24T23:53:08.153 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T23:53:08.154 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T23:53:08.721 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:53:08.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:08 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1551495439' entity='client.admin' 2024-04-24T23:53:09.751 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:09.751 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:09.751 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:53:09.751 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:09.751 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:09.751 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:10.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.209+0000 7f5de281a700 1 -- 172.21.15.110:0/1141986896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 msgr2=0x7f5ddc101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.209+0000 7f5de281a700 1 --2- 172.21.15.110:0/1141986896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 0x7f5ddc101020 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009910 tx=0x7f5dcc009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.209+0000 7f5de281a700 1 -- 172.21.15.110:0/1141986896 shutdown_connections 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.209+0000 7f5de281a700 1 --2- 172.21.15.110:0/1141986896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 0x7f5ddc101020 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.209+0000 7f5de281a700 1 -- 172.21.15.110:0/1141986896 >> 172.21.15.110:0/1141986896 conn(0x7f5ddc0fc1a0 msgr2=0x7f5ddc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.209+0000 7f5de281a700 1 -- 172.21.15.110:0/1141986896 shutdown_connections 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.209+0000 7f5de281a700 1 -- 172.21.15.110:0/1141986896 wait complete. 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5de281a700 1 Processor -- start 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5de281a700 1 -- start start 2024-04-24T23:53:10.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5de281a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 0x7f5ddc1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5de281a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc106ad0 con 0x7f5ddc100c00 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5ddbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 0x7f5ddc1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5ddbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 0x7f5ddc1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35822/0 (socket says 172.21.15.110:35822) 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5ddbfff700 1 -- 172.21.15.110:0/1586948720 learned_addr learned my addr 172.21.15.110:0/1586948720 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5ddbfff700 1 -- 172.21.15.110:0/1586948720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc197820 con 0x7f5ddc100c00 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.210+0000 7f5ddbfff700 1 --2- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 0x7f5ddc1972e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc005800 tx=0x7f5dcc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.211+0000 7f5dd97fa700 1 -- 172.21.15.110:0/1586948720 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dcc027070 con 0x7f5ddc100c00 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.211+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc197ab0 con 0x7f5ddc100c00 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.211+0000 7f5dd97fa700 1 -- 172.21.15.110:0/1586948720 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dcc00be10 con 0x7f5ddc100c00 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.211+0000 7f5dd97fa700 1 -- 172.21.15.110:0/1586948720 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dcc00f460 con 0x7f5ddc100c00 2024-04-24T23:53:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.211+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc19a7a0 con 0x7f5ddc100c00 2024-04-24T23:53:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.212+0000 7f5dd97fa700 1 -- 172.21.15.110:0/1586948720 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5dcc02a070 con 0x7f5ddc100c00 2024-04-24T23:53:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.212+0000 7f5dd97fa700 1 --2- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5dc40390f0 0x7f5dc403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.212+0000 7f5dd97fa700 1 -- 172.21.15.110:0/1586948720 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5dcc01e030 con 0x7f5ddc100c00 2024-04-24T23:53:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.212+0000 7f5ddb7fe700 1 --2- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5dc40390f0 0x7f5dc403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.212+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ddc107250 con 0x7f5ddc100c00 2024-04-24T23:53:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.212+0000 7f5ddb7fe700 1 --2- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5dc40390f0 0x7f5dc403b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0006fd0 tx=0x7f5dd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:10.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.217+0000 7f5dd97fa700 1 -- 172.21.15.110:0/1586948720 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5dcc031070 con 0x7f5ddc100c00 2024-04-24T23:53:10.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.417+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f5ddc062670 con 0x7f5dc40390f0 2024-04-24T23:53:10.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.420+0000 7f5dd97fa700 1 -- 172.21.15.110:0/1586948720 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ddc062670 con 0x7f5dc40390f0 2024-04-24T23:53:10.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.422+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5dc40390f0 msgr2=0x7f5dc403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:10.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.422+0000 7f5de281a700 1 --2- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5dc40390f0 0x7f5dc403b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0006fd0 tx=0x7f5dd0006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:10.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.422+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 msgr2=0x7f5ddc1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:10.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.422+0000 7f5de281a700 1 --2- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 0x7f5ddc1972e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc005800 tx=0x7f5dcc009f60 comp rx=0 tx=0).stop 2024-04-24T23:53:10.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.423+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 shutdown_connections 2024-04-24T23:53:10.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.423+0000 7f5de281a700 1 --2- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5dc40390f0 0x7f5dc403b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:10.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.423+0000 7f5de281a700 1 --2- 172.21.15.110:0/1586948720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ddc100c00 0x7f5ddc1972e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:10.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.423+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 >> 172.21.15.110:0/1586948720 conn(0x7f5ddc0fc1a0 msgr2=0x7f5ddc069aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:10.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.423+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 shutdown_connections 2024-04-24T23:53:10.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:10.423+0000 7f5de281a700 1 -- 172.21.15.110:0/1586948720 wait complete. 2024-04-24T23:53:11.157 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2024-04-24T23:53:11.157 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:53:11.157 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:53:11.175 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:53:11.176 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:53:11.233 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2024-04-24T23:53:11.233 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch host add smithi173 2024-04-24T23:53:11.469 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:11.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:12.731 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:12 smithi110 ceph-mon[29187]: Updating smithi110:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:12.731 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:12 smithi110 ceph-mon[29187]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:53:12.731 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:12 smithi110 ceph-mon[29187]: Updating smithi110:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:53:13.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.047+0000 7f95ee8ef700 1 -- 172.21.15.110:0/2053024572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 msgr2=0x7f95e8103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:13.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.047+0000 7f95ee8ef700 1 --2- 172.21.15.110:0/2053024572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 0x7f95e8103250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009910 tx=0x7f95d8009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:13.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.047+0000 7f95ee8ef700 1 -- 172.21.15.110:0/2053024572 shutdown_connections 2024-04-24T23:53:13.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.047+0000 7f95ee8ef700 1 --2- 172.21.15.110:0/2053024572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 0x7f95e8103250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:13.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.047+0000 7f95ee8ef700 1 -- 172.21.15.110:0/2053024572 >> 172.21.15.110:0/2053024572 conn(0x7f95e8077800 msgr2=0x7f95e80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:13.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.047+0000 7f95ee8ef700 1 -- 172.21.15.110:0/2053024572 shutdown_connections 2024-04-24T23:53:13.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.047+0000 7f95ee8ef700 1 -- 172.21.15.110:0/2053024572 wait complete. 2024-04-24T23:53:13.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.048+0000 7f95ee8ef700 1 Processor -- start 2024-04-24T23:53:13.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.048+0000 7f95ee8ef700 1 -- start start 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.048+0000 7f95ee8ef700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 0x7f95e81976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.048+0000 7f95ee8ef700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95e810c170 con 0x7f95e8102e70 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.048+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 0x7f95e81976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.048+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 0x7f95e81976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35840/0 (socket says 172.21.15.110:35840) 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.048+0000 7f95e7fff700 1 -- 172.21.15.110:0/1956664559 learned_addr learned my addr 172.21.15.110:0/1956664559 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.048+0000 7f95e7fff700 1 -- 172.21.15.110:0/1956664559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e8197c10 con 0x7f95e8102e70 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.049+0000 7f95e7fff700 1 --2- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 0x7f95e81976d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95d80056d0 tx=0x7f95d80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.049+0000 7f95e57fa700 1 -- 172.21.15.110:0/1956664559 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95d8026070 con 0x7f95e8102e70 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.049+0000 7f95e57fa700 1 -- 172.21.15.110:0/1956664559 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95d8004370 con 0x7f95e8102e70 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.049+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e8197ea0 con 0x7f95e8102e70 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.049+0000 7f95e57fa700 1 -- 172.21.15.110:0/1956664559 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95d80039b0 con 0x7f95e8102e70 2024-04-24T23:53:13.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.049+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e8198300 con 0x7f95e8102e70 2024-04-24T23:53:13.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.050+0000 7f95e57fa700 1 -- 172.21.15.110:0/1956664559 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f95d8029070 con 0x7f95e8102e70 2024-04-24T23:53:13.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.050+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e8069ea0 con 0x7f95e8102e70 2024-04-24T23:53:13.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.050+0000 7f95e57fa700 1 --2- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95d0039150 0x7f95d003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:13.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.050+0000 7f95e57fa700 1 -- 172.21.15.110:0/1956664559 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f95d801e070 con 0x7f95e8102e70 2024-04-24T23:53:13.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.050+0000 7f95e77fe700 1 --2- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95d0039150 0x7f95d003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:13.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.050+0000 7f95e77fe700 1 --2- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95d0039150 0x7f95d003b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f95dc006fd0 tx=0x7f95dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:13.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.055+0000 7f95e57fa700 1 -- 172.21.15.110:0/1956664559 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f95d8013070 con 0x7f95e8102e70 2024-04-24T23:53:13.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:13.253+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}) -- 0x7f95e8198aa0 con 0x7f95d0039150 2024-04-24T23:53:14.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:13 smithi110 ceph-mon[29187]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:53:15.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:15 smithi110 ceph-mon[29187]: Deploying cephadm binary to smithi173 2024-04-24T23:53:15.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:15 smithi110 ceph-mon[29187]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:17.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.288+0000 7f95e57fa700 1 -- 172.21.15.110:0/1956664559 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f95e8198aa0 con 0x7f95d0039150 2024-04-24T23:53:17.289 INFO:teuthology.orchestra.run.smithi110.stdout:Added host 'smithi173' with addr '172.21.15.173' 2024-04-24T23:53:17.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95d0039150 msgr2=0x7f95d003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:17.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 --2- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95d0039150 0x7f95d003b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f95dc006fd0 tx=0x7f95dc006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:17.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 msgr2=0x7f95e81976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:17.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 --2- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 0x7f95e81976d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95d80056d0 tx=0x7f95d80057b0 comp rx=0 tx=0).stop 2024-04-24T23:53:17.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 shutdown_connections 2024-04-24T23:53:17.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 --2- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95d0039150 0x7f95d003b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:17.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 --2- 172.21.15.110:0/1956664559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95e8102e70 0x7f95e81976d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:17.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 >> 172.21.15.110:0/1956664559 conn(0x7f95e8077800 msgr2=0x7f95e8103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:17.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 shutdown_connections 2024-04-24T23:53:17.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:17.291+0000 7f95ee8ef700 1 -- 172.21.15.110:0/1956664559 wait complete. 2024-04-24T23:53:17.605 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:17 smithi110 ceph-mon[29187]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:17.606 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:17 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:17.606 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:17 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:17.942 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:53:18.177 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:53:18.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:18 smithi110 ceph-mon[29187]: Added host smithi173 2024-04-24T23:53:18.618 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:18 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:18.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:18 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.659+0000 7fc4ac43b700 1 -- 172.21.15.110:0/1994371172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 msgr2=0x7fc4a4102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.659+0000 7fc4ac43b700 1 --2- 172.21.15.110:0/1994371172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 0x7fc4a4102250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc494009910 tx=0x7fc494009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.659+0000 7fc4ac43b700 1 -- 172.21.15.110:0/1994371172 shutdown_connections 2024-04-24T23:53:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.659+0000 7fc4ac43b700 1 --2- 172.21.15.110:0/1994371172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 0x7fc4a4102250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:19.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.659+0000 7fc4ac43b700 1 -- 172.21.15.110:0/1994371172 >> 172.21.15.110:0/1994371172 conn(0x7fc4a40fd740 msgr2=0x7fc4a40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:19.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.659+0000 7fc4ac43b700 1 -- 172.21.15.110:0/1994371172 shutdown_connections 2024-04-24T23:53:19.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.659+0000 7fc4ac43b700 1 -- 172.21.15.110:0/1994371172 wait complete. 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.660+0000 7fc4ac43b700 1 Processor -- start 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.660+0000 7fc4ac43b700 1 -- start start 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.660+0000 7fc4ac43b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 0x7fc4a4197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.660+0000 7fc4ac43b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a410c0f0 con 0x7fc4a4101e70 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.660+0000 7fc4aa1d7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 0x7fc4a4197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.660+0000 7fc4aa1d7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 0x7fc4a4197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56200/0 (socket says 172.21.15.110:56200) 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.660+0000 7fc4aa1d7700 1 -- 172.21.15.110:0/3139436618 learned_addr learned my addr 172.21.15.110:0/3139436618 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.661+0000 7fc4aa1d7700 1 -- 172.21.15.110:0/3139436618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a4197b90 con 0x7fc4a4101e70 2024-04-24T23:53:19.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.661+0000 7fc4aa1d7700 1 --2- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 0x7fc4a4197650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc494006010 tx=0x7fc494005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:19.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.661+0000 7fc49b7fe700 1 -- 172.21.15.110:0/3139436618 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc494026070 con 0x7fc4a4101e70 2024-04-24T23:53:19.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.661+0000 7fc49b7fe700 1 -- 172.21.15.110:0/3139436618 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc49400bba0 con 0x7fc4a4101e70 2024-04-24T23:53:19.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.661+0000 7fc49b7fe700 1 -- 172.21.15.110:0/3139436618 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4940038a0 con 0x7fc4a4101e70 2024-04-24T23:53:19.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.661+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a4197e20 con 0x7fc4a4101e70 2024-04-24T23:53:19.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.661+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a419bf20 con 0x7fc4a4101e70 2024-04-24T23:53:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.662+0000 7fc49b7fe700 1 -- 172.21.15.110:0/3139436618 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc494029070 con 0x7fc4a4101e70 2024-04-24T23:53:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.662+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a4103350 con 0x7fc4a4101e70 2024-04-24T23:53:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.662+0000 7fc49b7fe700 1 --2- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc49003d560 0x7fc49003fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.662+0000 7fc49b7fe700 1 -- 172.21.15.110:0/3139436618 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc49401e070 con 0x7fc4a4101e70 2024-04-24T23:53:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.662+0000 7fc4a99d6700 1 --2- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc49003d560 0x7fc49003fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:19.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.662+0000 7fc4a99d6700 1 --2- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc49003d560 0x7fc49003fa20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:19.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.667+0000 7fc49b7fe700 1 -- 172.21.15.110:0/3139436618 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc494013070 con 0x7fc4a4101e70 2024-04-24T23:53:19.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:19 smithi110 ceph-mon[29187]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:19.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.871+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4a41989b0 con 0x7fc49003d560 2024-04-24T23:53:19.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.872+0000 7fc49b7fe700 1 -- 172.21.15.110:0/3139436618 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fc4a41989b0 con 0x7fc49003d560 2024-04-24T23:53:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:19.872 INFO:teuthology.orchestra.run.smithi110.stdout:[{"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2024-04-24T23:53:19.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc49003d560 msgr2=0x7fc49003fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 --2- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc49003d560 0x7fc49003fa20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 msgr2=0x7fc4a4197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 --2- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 0x7fc4a4197650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc494006010 tx=0x7fc494005780 comp rx=0 tx=0).stop 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 shutdown_connections 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 --2- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc49003d560 0x7fc49003fa20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 --2- 172.21.15.110:0/3139436618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4a4101e70 0x7fc4a4197650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 >> 172.21.15.110:0/3139436618 conn(0x7fc4a40fd740 msgr2=0x7fc4a4103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 shutdown_connections 2024-04-24T23:53:19.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:19.874+0000 7fc4ac43b700 1 -- 172.21.15.110:0/3139436618 wait complete. 2024-04-24T23:53:20.473 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi188 2024-04-24T23:53:20.474 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:53:20.474 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:53:20.492 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:53:20.492 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:53:20.550 INFO:tasks.cephadm:Adding host smithi188 to orchestrator... 2024-04-24T23:53:20.550 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch host add smithi188 2024-04-24T23:53:20.785 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:53:21.196 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:20 smithi110 ceph-mon[29187]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:53:21.196 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:20 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:21.197 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:20 smithi110 ceph-mon[29187]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:22.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.284+0000 7f3534327700 1 -- 172.21.15.110:0/954057692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 msgr2=0x7f352c102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:22.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.284+0000 7f3534327700 1 --2- 172.21.15.110:0/954057692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 0x7f352c102250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f351c009910 tx=0x7f351c009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:22.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.284+0000 7f3534327700 1 -- 172.21.15.110:0/954057692 shutdown_connections 2024-04-24T23:53:22.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.284+0000 7f3534327700 1 --2- 172.21.15.110:0/954057692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 0x7f352c102250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:22.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.284+0000 7f3534327700 1 -- 172.21.15.110:0/954057692 >> 172.21.15.110:0/954057692 conn(0x7f352c0fd740 msgr2=0x7f352c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:22.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.284+0000 7f3534327700 1 -- 172.21.15.110:0/954057692 shutdown_connections 2024-04-24T23:53:22.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.284+0000 7f3534327700 1 -- 172.21.15.110:0/954057692 wait complete. 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.285+0000 7f3534327700 1 Processor -- start 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.285+0000 7f3534327700 1 -- start start 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.285+0000 7f3534327700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 0x7f352c074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.285+0000 7f3534327700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f352c10c0f0 con 0x7f352c101e70 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.285+0000 7f35320c3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 0x7f352c074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.285+0000 7f35320c3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 0x7f352c074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56222/0 (socket says 172.21.15.110:56222) 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.285+0000 7f35320c3700 1 -- 172.21.15.110:0/3102364895 learned_addr learned my addr 172.21.15.110:0/3102364895 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.286+0000 7f35320c3700 1 -- 172.21.15.110:0/3102364895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c076420 con 0x7f352c101e70 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.286+0000 7f35320c3700 1 --2- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 0x7f352c074530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f351c0056d0 tx=0x7f351c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.286+0000 7f35237fe700 1 -- 172.21.15.110:0/3102364895 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f351c026070 con 0x7f352c101e70 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.286+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c074ad0 con 0x7f352c101e70 2024-04-24T23:53:22.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.286+0000 7f35237fe700 1 -- 172.21.15.110:0/3102364895 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351c00bc00 con 0x7f352c101e70 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.286+0000 7f35237fe700 1 -- 172.21.15.110:0/3102364895 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f351c0039b0 con 0x7f352c101e70 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.286+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c074f90 con 0x7f352c101e70 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.287+0000 7f35237fe700 1 -- 172.21.15.110:0/3102364895 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f351c029070 con 0x7f352c101e70 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.287+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352c103350 con 0x7f352c101e70 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.287+0000 7f35237fe700 1 --2- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35180390f0 0x7f351803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.287+0000 7f35237fe700 1 -- 172.21.15.110:0/3102364895 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f351c01e070 con 0x7f352c101e70 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.287+0000 7f35318c2700 1 --2- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35180390f0 0x7f351803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.287+0000 7f35318c2700 1 --2- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35180390f0 0x7f351803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3528006fd0 tx=0x7f3528006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.292+0000 7f35237fe700 1 -- 172.21.15.110:0/3102364895 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f351c013070 con 0x7f352c101e70 2024-04-24T23:53:22.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:22.493+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}) -- 0x7f352c075a60 con 0x7f35180390f0 2024-04-24T23:53:23.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:23.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:53:23.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:23.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:23.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:23.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:23.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:53:23.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:23.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:24.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:24 smithi110 ceph-mon[29187]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-24T23:53:24.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:24 smithi110 ceph-mon[29187]: Updating smithi173:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:24.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:24 smithi110 ceph-mon[29187]: Deploying cephadm binary to smithi188 2024-04-24T23:53:24.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:24 smithi110 ceph-mon[29187]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:53:24.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:24 smithi110 ceph-mon[29187]: Updating smithi173:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:53:25.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:25 smithi110 ceph-mon[29187]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:26.715 INFO:teuthology.orchestra.run.smithi110.stdout:Added host 'smithi188' with addr '172.21.15.188' 2024-04-24T23:53:26.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.714+0000 7f35237fe700 1 -- 172.21.15.110:0/3102364895 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f352c075a60 con 0x7f35180390f0 2024-04-24T23:53:26.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35180390f0 msgr2=0x7f351803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:26.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 --2- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35180390f0 0x7f351803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3528006fd0 tx=0x7f3528006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:26.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 msgr2=0x7f352c074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:26.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 --2- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 0x7f352c074530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f351c0056d0 tx=0x7f351c0057b0 comp rx=0 tx=0).stop 2024-04-24T23:53:26.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 shutdown_connections 2024-04-24T23:53:26.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 --2- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35180390f0 0x7f351803b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:26.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 --2- 172.21.15.110:0/3102364895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f352c101e70 0x7f352c074530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:26.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 >> 172.21.15.110:0/3102364895 conn(0x7f352c0fd740 msgr2=0x7f352c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:26.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 shutdown_connections 2024-04-24T23:53:26.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:26.717+0000 7f3534327700 1 -- 172.21.15.110:0/3102364895 wait complete. 2024-04-24T23:53:26.976 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:26.976 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:26.976 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:26.977 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:26 smithi110 ceph-mon[29187]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:26.977 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:26.977 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:27.283 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:53:27.514 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:53:28.512 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:28 smithi110 ceph-mon[29187]: Added host smithi188 2024-04-24T23:53:28.512 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:28.512 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:29.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.024+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/4064709508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 msgr2=0x7f8bf8106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.024+0000 7f8bfd6e0700 1 --2- 172.21.15.110:0/4064709508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 0x7f8bf8106fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009910 tx=0x7f8bf0009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.024+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/4064709508 shutdown_connections 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.024+0000 7f8bfd6e0700 1 --2- 172.21.15.110:0/4064709508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 0x7f8bf8106fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.024+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/4064709508 >> 172.21.15.110:0/4064709508 conn(0x7f8bf8077800 msgr2=0x7f8bf80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.024+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/4064709508 shutdown_connections 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.024+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/4064709508 wait complete. 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bfd6e0700 1 Processor -- start 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bfd6e0700 1 -- start start 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bfd6e0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 0x7f8bf8197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bfd6e0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf810b050 con 0x7f8bf8106be0 2024-04-24T23:53:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bf6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 0x7f8bf8197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bf6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 0x7f8bf8197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46474/0 (socket says 172.21.15.110:46474) 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bf6ffd700 1 -- 172.21.15.110:0/236000828 learned_addr learned my addr 172.21.15.110:0/236000828 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bf6ffd700 1 -- 172.21.15.110:0/236000828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf8197bb0 con 0x7f8bf8106be0 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.025+0000 7f8bf6ffd700 1 --2- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 0x7f8bf8197670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00056d0 tx=0x7f8bf00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.026+0000 7f8bdffff700 1 -- 172.21.15.110:0/236000828 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf0026070 con 0x7f8bf8106be0 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.026+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf8197e40 con 0x7f8bf8106be0 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.026+0000 7f8bdffff700 1 -- 172.21.15.110:0/236000828 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bf000bc00 con 0x7f8bf8106be0 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.026+0000 7f8bdffff700 1 -- 172.21.15.110:0/236000828 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf00039b0 con 0x7f8bf8106be0 2024-04-24T23:53:29.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.026+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf819bf40 con 0x7f8bf8106be0 2024-04-24T23:53:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.027+0000 7f8bdffff700 1 -- 172.21.15.110:0/236000828 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8bf0029070 con 0x7f8bf8106be0 2024-04-24T23:53:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.027+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf81021e0 con 0x7f8bf8106be0 2024-04-24T23:53:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.027+0000 7f8bdffff700 1 --2- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be00390f0 0x7f8be003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.027+0000 7f8bdffff700 1 -- 172.21.15.110:0/236000828 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8bf001e070 con 0x7f8bf8106be0 2024-04-24T23:53:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.027+0000 7f8bf67fc700 1 --2- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be00390f0 0x7f8be003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.027+0000 7f8bf67fc700 1 --2- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be00390f0 0x7f8be003b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bec006fd0 tx=0x7f8bec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.032+0000 7f8bdffff700 1 -- 172.21.15.110:0/236000828 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8bf0013070 con 0x7f8bf8106be0 2024-04-24T23:53:29.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.234+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8bf81989d0 con 0x7f8be00390f0 2024-04-24T23:53:29.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.235+0000 7f8bdffff700 1 -- 172.21.15.110:0/236000828 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f8bf81989d0 con 0x7f8be00390f0 2024-04-24T23:53:29.234 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:53:29.235 INFO:teuthology.orchestra.run.smithi110.stdout:[{"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}, {"addr": "172.21.15.188", "hostname": "smithi188", "labels": [], "status": ""}] 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be00390f0 msgr2=0x7f8be003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 --2- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be00390f0 0x7f8be003b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bec006fd0 tx=0x7f8bec006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 msgr2=0x7f8bf8197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 --2- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 0x7f8bf8197670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00056d0 tx=0x7f8bf00057b0 comp rx=0 tx=0).stop 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 shutdown_connections 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 --2- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be00390f0 0x7f8be003b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 --2- 172.21.15.110:0/236000828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bf8106be0 0x7f8bf8197670 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 >> 172.21.15.110:0/236000828 conn(0x7f8bf8077800 msgr2=0x7f8bf8069660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 shutdown_connections 2024-04-24T23:53:29.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:29.237+0000 7f8bfd6e0700 1 -- 172.21.15.110:0/236000828 wait complete. 2024-04-24T23:53:29.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:29 smithi110 ceph-mon[29187]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:29.904 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T23:53:29.904 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T23:53:30.135 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:53:30.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:30 smithi110 ceph-mon[29187]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:53:30.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:30 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:31.462 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:31 smithi110 ceph-mon[29187]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.688+0000 7f3b22bb1700 1 -- 172.21.15.110:0/1568446042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 msgr2=0x7f3b1c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.688+0000 7f3b22bb1700 1 --2- 172.21.15.110:0/1568446042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 0x7f3b1c107050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009910 tx=0x7f3b10009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.688+0000 7f3b22bb1700 1 -- 172.21.15.110:0/1568446042 shutdown_connections 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.688+0000 7f3b22bb1700 1 --2- 172.21.15.110:0/1568446042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 0x7f3b1c107050 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.688+0000 7f3b22bb1700 1 -- 172.21.15.110:0/1568446042 >> 172.21.15.110:0/1568446042 conn(0x7f3b1c0fbdb0 msgr2=0x7f3b1c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.688+0000 7f3b22bb1700 1 -- 172.21.15.110:0/1568446042 shutdown_connections 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.688+0000 7f3b22bb1700 1 -- 172.21.15.110:0/1568446042 wait complete. 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.689+0000 7f3b22bb1700 1 Processor -- start 2024-04-24T23:53:31.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.689+0000 7f3b22bb1700 1 -- start start 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.689+0000 7f3b22bb1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 0x7f3b1c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.689+0000 7f3b22bb1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c10c280 con 0x7f3b1c106c70 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.689+0000 7f3b2094d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 0x7f3b1c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.689+0000 7f3b2094d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 0x7f3b1c1976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46496/0 (socket says 172.21.15.110:46496) 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.689+0000 7f3b2094d700 1 -- 172.21.15.110:0/3530717928 learned_addr learned my addr 172.21.15.110:0/3530717928 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.690+0000 7f3b2094d700 1 -- 172.21.15.110:0/3530717928 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c197bf0 con 0x7f3b1c106c70 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.690+0000 7f3b2094d700 1 --2- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 0x7f3b1c1976b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3b100056d0 tx=0x7f3b100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.690+0000 7f3b19ffb700 1 -- 172.21.15.110:0/3530717928 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b10026070 con 0x7f3b1c106c70 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.690+0000 7f3b19ffb700 1 -- 172.21.15.110:0/3530717928 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b1000bc00 con 0x7f3b1c106c70 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.690+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1c197e80 con 0x7f3b1c106c70 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.690+0000 7f3b19ffb700 1 -- 172.21.15.110:0/3530717928 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b100039b0 con 0x7f3b1c106c70 2024-04-24T23:53:31.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.690+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1c1982e0 con 0x7f3b1c106c70 2024-04-24T23:53:31.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.691+0000 7f3b19ffb700 1 -- 172.21.15.110:0/3530717928 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3b10029070 con 0x7f3b1c106c70 2024-04-24T23:53:31.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.691+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1c1033b0 con 0x7f3b1c106c70 2024-04-24T23:53:31.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.691+0000 7f3b19ffb700 1 --2- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b0c0390f0 0x7f3b0c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:31.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.691+0000 7f3b19ffb700 1 -- 172.21.15.110:0/3530717928 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3b1001e070 con 0x7f3b1c106c70 2024-04-24T23:53:31.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.691+0000 7f3b1bfff700 1 --2- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b0c0390f0 0x7f3b0c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:31.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.692+0000 7f3b1bfff700 1 --2- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b0c0390f0 0x7f3b0c03b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3b08006fd0 tx=0x7f3b08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:31.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.696+0000 7f3b19ffb700 1 -- 172.21.15.110:0/3530717928 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b10013070 con 0x7f3b1c106c70 2024-04-24T23:53:31.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:31.902+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3b1c198a10 con 0x7f3b1c106c70 2024-04-24T23:53:32.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.270+0000 7f3b19ffb700 1 -- 172.21.15.110:0/3530717928 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3b10030020 con 0x7f3b1c106c70 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b0c0390f0 msgr2=0x7f3b0c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 --2- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b0c0390f0 0x7f3b0c03b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3b08006fd0 tx=0x7f3b08006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 msgr2=0x7f3b1c1976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 --2- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 0x7f3b1c1976b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3b100056d0 tx=0x7f3b100057b0 comp rx=0 tx=0).stop 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 shutdown_connections 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 --2- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b0c0390f0 0x7f3b0c03b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 --2- 172.21.15.110:0/3530717928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b1c106c70 0x7f3b1c1976b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 >> 172.21.15.110:0/3530717928 conn(0x7f3b1c0fbdb0 msgr2=0x7f3b1c1097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 shutdown_connections 2024-04-24T23:53:32.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:32.273+0000 7f3b22bb1700 1 -- 172.21.15.110:0/3530717928 wait complete. 2024-04-24T23:53:32.274 INFO:teuthology.orchestra.run.smithi110.stderr:adjusted tunables profile to default 2024-04-24T23:53:32.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:32 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3530717928' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:53:32.906 INFO:tasks.cephadm:Adding mon.a on smithi110 2024-04-24T23:53:32.906 INFO:tasks.cephadm:Adding mon.b on smithi173 2024-04-24T23:53:32.906 INFO:tasks.cephadm:Adding mon.c on smithi188 2024-04-24T23:53:32.907 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi110:172.21.15.110=a;smithi173:172.21.15.173=b;smithi188:172.21.15.188=c' 2024-04-24T23:53:33.139 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:33.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3530717928' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: Updating smithi188:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:33.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:33.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:33.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:34.676 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 -- 172.21.15.188:0/898495663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 msgr2=0x7fd0281031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:34.676 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 --2- 172.21.15.188:0/898495663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 0x7fd0281031e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd024009910 tx=0x7fd024009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:34.676 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 -- 172.21.15.188:0/898495663 shutdown_connections 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 --2- 172.21.15.188:0/898495663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 0x7fd0281031e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 -- 172.21.15.188:0/898495663 >> 172.21.15.188:0/898495663 conn(0x7fd0280fc630 msgr2=0x7fd0280fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 -- 172.21.15.188:0/898495663 shutdown_connections 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 -- 172.21.15.188:0/898495663 wait complete. 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 Processor -- start 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.675+0000 7fd02f520700 1 -- start start 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.676+0000 7fd02f520700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 0x7fd02819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.676+0000 7fd02f520700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd02810c1a0 con 0x7fd028102e00 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.676+0000 7fd02d2bc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 0x7fd02819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.676+0000 7fd02d2bc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 0x7fd02819d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.188:57780/0 (socket says 172.21.15.188:57780) 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.676+0000 7fd02d2bc700 1 -- 172.21.15.188:0/807242920 learned_addr learned my addr 172.21.15.188:0/807242920 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:53:34.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.676+0000 7fd02d2bc700 1 -- 172.21.15.188:0/807242920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02819d910 con 0x7fd028102e00 2024-04-24T23:53:34.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.676+0000 7fd02d2bc700 1 --2- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 0x7fd02819d3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd0240056d0 tx=0x7fd0240057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:34.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.677+0000 7fd01e7fc700 1 -- 172.21.15.188:0/807242920 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd024026070 con 0x7fd028102e00 2024-04-24T23:53:34.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.677+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02819dba0 con 0x7fd028102e00 2024-04-24T23:53:34.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.677+0000 7fd01e7fc700 1 -- 172.21.15.188:0/807242920 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd024004370 con 0x7fd028102e00 2024-04-24T23:53:34.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.677+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd028197590 con 0x7fd028102e00 2024-04-24T23:53:34.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.677+0000 7fd01e7fc700 1 -- 172.21.15.188:0/807242920 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0240039a0 con 0x7fd028102e00 2024-04-24T23:53:34.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.677+0000 7fd01e7fc700 1 -- 172.21.15.188:0/807242920 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd024029070 con 0x7fd028102e00 2024-04-24T23:53:34.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.677+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02806a930 con 0x7fd028102e00 2024-04-24T23:53:34.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.678+0000 7fd01e7fc700 1 --2- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0140390f0 0x7fd01403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:34.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.678+0000 7fd01e7fc700 1 -- 172.21.15.188:0/807242920 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd02401e070 con 0x7fd028102e00 2024-04-24T23:53:34.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.678+0000 7fd02cabb700 1 --2- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0140390f0 0x7fd01403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:34.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.678+0000 7fd02cabb700 1 --2- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0140390f0 0x7fd01403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd018006fd0 tx=0x7fd018006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:34.682 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.683+0000 7fd01e7fc700 1 -- 172.21.15.188:0/807242920 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd024013070 con 0x7fd028102e00 2024-04-24T23:53:34.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:34 smithi110 ceph-mon[29187]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:53:34.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:34 smithi110 ceph-mon[29187]: Updating smithi188:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:53:34.888 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.888+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi110:172.21.15.110=a;smithi173:172.21.15.173=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}) -- 0x7fd028198ac0 con 0x7fd0140390f0 2024-04-24T23:53:34.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.895+0000 7fd01e7fc700 1 -- 172.21.15.188:0/807242920 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd028198ac0 con 0x7fd0140390f0 2024-04-24T23:53:34.895 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mon update... 2024-04-24T23:53:34.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0140390f0 msgr2=0x7fd01403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:34.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 --2- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0140390f0 0x7fd01403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd018006fd0 tx=0x7fd018006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:34.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 msgr2=0x7fd02819d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:34.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 --2- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 0x7fd02819d3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd0240056d0 tx=0x7fd0240057b0 comp rx=0 tx=0).stop 2024-04-24T23:53:34.897 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 shutdown_connections 2024-04-24T23:53:34.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 --2- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0140390f0 0x7fd01403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:34.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 --2- 172.21.15.188:0/807242920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd028102e00 0x7fd02819d3d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:34.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 >> 172.21.15.188:0/807242920 conn(0x7fd0280fc630 msgr2=0x7fd028105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:34.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 shutdown_connections 2024-04-24T23:53:34.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:34.897+0000 7fd02f520700 1 -- 172.21.15.188:0/807242920 wait complete. 2024-04-24T23:53:35.573 DEBUG:teuthology.orchestra.run.smithi173:mon.b> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T23:53:35.576 DEBUG:teuthology.orchestra.run.smithi188:mon.c> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T23:53:35.578 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:53:35.578 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:53:35.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:35 smithi110 ceph-mon[29187]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:35.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:35.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:35.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:35.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:35.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:35.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:53:35.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:36.340 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:36.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:36 smithi110 ceph-mon[29187]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi110:172.21.15.110=a;smithi173:172.21.15.173=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:53:36.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:36 smithi110 ceph-mon[29187]: Saving service mon spec with placement smithi110:172.21.15.110=a;smithi173:172.21.15.173=b;smithi188:172.21.15.188=c;count:3 2024-04-24T23:53:36.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:36 smithi110 ceph-mon[29187]: Deploying daemon mon.c on smithi188 2024-04-24T23:53:37.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.779+0000 7fbd1667c700 1 -- 172.21.15.188:0/1956625677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 msgr2=0x7fbd10106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:37.779 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.779+0000 7fbd1667c700 1 --2- 172.21.15.188:0/1956625677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 0x7fbd10106fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009910 tx=0x7fbd00009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.779+0000 7fbd1667c700 1 -- 172.21.15.188:0/1956625677 shutdown_connections 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.779+0000 7fbd1667c700 1 --2- 172.21.15.188:0/1956625677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 0x7fbd10106fe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.779+0000 7fbd1667c700 1 -- 172.21.15.188:0/1956625677 >> 172.21.15.188:0/1956625677 conn(0x7fbd100fc620 msgr2=0x7fbd100fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.779+0000 7fbd1667c700 1 -- 172.21.15.188:0/1956625677 shutdown_connections 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.779+0000 7fbd1667c700 1 -- 172.21.15.188:0/1956625677 wait complete. 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.780+0000 7fbd1667c700 1 Processor -- start 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.780+0000 7fbd1667c700 1 -- start start 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.780+0000 7fbd1667c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 0x7fbd10197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:37.780 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.780+0000 7fbd1667c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1010b070 con 0x7fbd10106c00 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.780+0000 7fbd0ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 0x7fbd10197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.780+0000 7fbd0ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 0x7fbd10197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.188:57800/0 (socket says 172.21.15.188:57800) 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.780+0000 7fbd0ffff700 1 -- 172.21.15.188:0/1135174452 learned_addr learned my addr 172.21.15.188:0/1135174452 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.781+0000 7fbd0ffff700 1 -- 172.21.15.188:0/1135174452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd10197b50 con 0x7fbd10106c00 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.781+0000 7fbd0ffff700 1 --2- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 0x7fbd10197610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd000056d0 tx=0x7fbd000057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.781+0000 7fbd0d7fa700 1 -- 172.21.15.188:0/1135174452 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd00026070 con 0x7fbd10106c00 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.781+0000 7fbd0d7fa700 1 -- 172.21.15.188:0/1135174452 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd0000bc00 con 0x7fbd10106c00 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.781+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd10197de0 con 0x7fbd10106c00 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.781+0000 7fbd0d7fa700 1 -- 172.21.15.188:0/1135174452 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd000039b0 con 0x7fbd10106c00 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.781+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1019bee0 con 0x7fbd10106c00 2024-04-24T23:53:37.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.782+0000 7fbd0d7fa700 1 -- 172.21.15.188:0/1135174452 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbd00029070 con 0x7fbd10106c00 2024-04-24T23:53:37.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.782+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd10102200 con 0x7fbd10106c00 2024-04-24T23:53:37.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.782+0000 7fbd0d7fa700 1 --2- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbcf80390f0 0x7fbcf803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:37.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.782+0000 7fbd0d7fa700 1 -- 172.21.15.188:0/1135174452 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbd0001e070 con 0x7fbd10106c00 2024-04-24T23:53:37.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.782+0000 7fbd0f7fe700 1 --2- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbcf80390f0 0x7fbcf803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:37.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.783+0000 7fbd0f7fe700 1 --2- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbcf80390f0 0x7fbcf803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd04006fd0 tx=0x7fbd04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:37.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:37 smithi110 ceph-mon[29187]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:37.787 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:37.787+0000 7fbd0d7fa700 1 -- 172.21.15.188:0/1135174452 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbd00013070 con 0x7fbd10106c00 2024-04-24T23:53:38.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.084+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbd10198970 con 0x7fbd10106c00 2024-04-24T23:53:38.085 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.085+0000 7fbd0d7fa700 1 -- 172.21.15.188:0/1135174452 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fbd00030020 con 0x7fbd10106c00 2024-04-24T23:53:38.085 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:53:38.085 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","modified":"2024-04-24T23:51:50.484996Z","created":"2024-04-24T23:51:50.484996Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:53:38.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.087+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbcf80390f0 msgr2=0x7fbcf803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:38.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.087+0000 7fbd1667c700 1 --2- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbcf80390f0 0x7fbcf803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd04006fd0 tx=0x7fbd04006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:38.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.087+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 msgr2=0x7fbd10197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:38.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.087+0000 7fbd1667c700 1 --2- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 0x7fbd10197610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd000056d0 tx=0x7fbd000057b0 comp rx=0 tx=0).stop 2024-04-24T23:53:38.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.087+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 shutdown_connections 2024-04-24T23:53:38.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.087+0000 7fbd1667c700 1 --2- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbcf80390f0 0x7fbcf803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:38.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.087+0000 7fbd1667c700 1 --2- 172.21.15.188:0/1135174452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd10106c00 0x7fbd10197610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:38.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.087+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 >> 172.21.15.188:0/1135174452 conn(0x7fbd100fc620 msgr2=0x7fbd1006a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:38.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.088+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 shutdown_connections 2024-04-24T23:53:38.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:38.088+0000 7fbd1667c700 1 -- 172.21.15.188:0/1135174452 wait complete. 2024-04-24T23:53:38.088 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2024-04-24T23:53:38.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:38 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/1135174452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:53:39.704 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:53:39.704 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:53:39.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:39 smithi110 ceph-mon[29187]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:40.076 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:53:41.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:40 smithi188 systemd[1]: Starting Ceph mon.c for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:53:41.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.780+0000 7f27d1f93700 1 -- 172.21.15.188:0/2045737157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 msgr2=0x7f27cc1081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:41.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.780+0000 7f27d1f93700 1 --2- 172.21.15.188:0/2045737157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 0x7f27cc1081d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009910 tx=0x7f27bc009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.780+0000 7f27d1f93700 1 -- 172.21.15.188:0/2045737157 shutdown_connections 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.780+0000 7f27d1f93700 1 --2- 172.21.15.188:0/2045737157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 0x7f27cc1081d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.780+0000 7f27d1f93700 1 -- 172.21.15.188:0/2045737157 >> 172.21.15.188:0/2045737157 conn(0x7f27cc075f90 msgr2=0x7f27cc0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.780+0000 7f27d1f93700 1 -- 172.21.15.188:0/2045737157 shutdown_connections 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.780+0000 7f27d1f93700 1 -- 172.21.15.188:0/2045737157 wait complete. 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.781+0000 7f27d1f93700 1 Processor -- start 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.781+0000 7f27d1f93700 1 -- start start 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.781+0000 7f27d1f93700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 0x7f27cc19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.781+0000 7f27d1f93700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc10c260 con 0x7f27cc107df0 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.782+0000 7f27cb7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 0x7f27cc19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.782+0000 7f27cb7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 0x7f27cc19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.188:57816/0 (socket says 172.21.15.188:57816) 2024-04-24T23:53:41.782 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.782+0000 7f27cb7fe700 1 -- 172.21.15.188:0/707977540 learned_addr learned my addr 172.21.15.188:0/707977540 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.782+0000 7f27cb7fe700 1 -- 172.21.15.188:0/707977540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc19d910 con 0x7f27cc107df0 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.782+0000 7f27cb7fe700 1 --2- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 0x7f27cc19d3d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0056d0 tx=0x7f27bc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.782+0000 7f27c97fa700 1 -- 172.21.15.188:0/707977540 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27bc026070 con 0x7f27cc107df0 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.782+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc19dba0 con 0x7f27cc107df0 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.783+0000 7f27c97fa700 1 -- 172.21.15.188:0/707977540 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27bc00bc00 con 0x7f27cc107df0 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.783+0000 7f27c97fa700 1 -- 172.21.15.188:0/707977540 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27bc0039b0 con 0x7f27cc107df0 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.783+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc197590 con 0x7f27cc107df0 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.783+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27cc06a150 con 0x7f27cc107df0 2024-04-24T23:53:41.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.783+0000 7f27c97fa700 1 -- 172.21.15.188:0/707977540 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f27bc029070 con 0x7f27cc107df0 2024-04-24T23:53:41.785 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.784+0000 7f27c97fa700 1 --2- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27b80390f0 0x7f27b803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:41.785 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.784+0000 7f27c97fa700 1 -- 172.21.15.188:0/707977540 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f27bc01e070 con 0x7f27cc107df0 2024-04-24T23:53:41.785 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.784+0000 7f27c2ffd700 1 --2- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27b80390f0 0x7f27b803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:41.785 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.784+0000 7f27c2ffd700 1 --2- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27b80390f0 0x7f27b803b5b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27b4006fd0 tx=0x7f27b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:41.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:41 smithi110 ceph-mon[29187]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:41.789 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:41.789+0000 7f27c97fa700 1 -- 172.21.15.188:0/707977540 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f27bc013070 con 0x7f27cc107df0 2024-04-24T23:53:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:41 smithi188 podman[33252]: 2024-04-24 23:53:41.815541031 +0000 UTC m=+0.020800146 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:53:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:41 smithi188 podman[33252]: 2024-04-24 23:53:41.941396702 +0000 UTC m=+0.146655800 container create c462243bdd476cc2be0ef2813924cdfc06b015b89f38503519b70cab25c80685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:53:42.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.081+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f27cc063890 con 0x7f27cc107df0 2024-04-24T23:53:42.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.082+0000 7f27c97fa700 1 -- 172.21.15.188:0/707977540 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f27bc030020 con 0x7f27cc107df0 2024-04-24T23:53:42.081 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:53:42.081 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","modified":"2024-04-24T23:51:50.484996Z","created":"2024-04-24T23:51:50.484996Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:53:42.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27b80390f0 msgr2=0x7f27b803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:42.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 --2- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27b80390f0 0x7f27b803b5b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27b4006fd0 tx=0x7f27b4006e40 comp rx=0 tx=0).stop 2024-04-24T23:53:42.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 msgr2=0x7f27cc19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:42.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 --2- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 0x7f27cc19d3d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0056d0 tx=0x7f27bc0057b0 comp rx=0 tx=0).stop 2024-04-24T23:53:42.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 shutdown_connections 2024-04-24T23:53:42.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 --2- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27b80390f0 0x7f27b803b5b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:42.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 --2- 172.21.15.188:0/707977540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27cc107df0 0x7f27cc19d3d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:42.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 >> 172.21.15.188:0/707977540 conn(0x7f27cc075f90 msgr2=0x7f27cc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:42.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 shutdown_connections 2024-04-24T23:53:42.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:42.084+0000 7f27d1f93700 1 -- 172.21.15.188:0/707977540 wait complete. 2024-04-24T23:53:42.084 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2024-04-24T23:53:42.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 podman[33252]: 2024-04-24 23:53:42.184357869 +0000 UTC m=+0.389616979 container init c462243bdd476cc2be0ef2813924cdfc06b015b89f38503519b70cab25c80685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 podman[33252]: 2024-04-24 23:53:42.187944977 +0000 UTC m=+0.393204077 container start c462243bdd476cc2be0ef2813924cdfc06b015b89f38503519b70cab25c80685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pidfile_write: ignore empty --pid-file 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: load: jerasure load: lrc 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Git sha 0 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: DB SUMMARY 2024-04-24T23:53:42.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: DB Session ID: 63YF02C3NMMMICXQAFRH 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:53:42.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.env: 0x556a602d1c60 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.info_log: 0x556a61019aa0 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.statistics: (nil) 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.use_fsync: 0 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:53:42.520 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.db_log_dir: 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.wal_dir: 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:53:42.521 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.write_buffer_manager: 0x556a60fec1e0 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.unordered_write: 0 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:53:42.522 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:53:42.523 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:53:42.523 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:53:42.523 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.row_cache: None 2024-04-24T23:53:42.523 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.wal_filter: None 2024-04-24T23:53:42.524 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.wal_compression: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:53:42.525 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:53:42.526 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_open_files: -1 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Compression algorithms supported: 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: kZSTD supported: 0 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:53:42.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.merge_operator: 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_filter: None 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556a61019ba0) 2024-04-24T23:53:42.528 INFO:journalctl@ceph.mon.c.smithi188.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_type: 0 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: data_block_index_type: 0 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_shortening: 1 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: checksum: 4 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: no_block_cache: 0 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache: 0x556a6101d090 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_options: 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: capacity : 536870912 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: num_shard_bits : 4 2024-04-24T23:53:42.529 INFO:journalctl@ceph.mon.c.smithi188.stdout: strict_capacity_limit : 0 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_compressed: (nil) 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: persistent_cache: (nil) 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_size: 4096 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_size_deviation: 10 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_restart_interval: 16 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_block_restart_interval: 1 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: metadata_block_size: 4096 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: partition_filters: 0 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: use_delta_encoding: 1 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: filter_policy: bloomfilter 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: whole_key_filtering: 1 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: verify_compression: 0 2024-04-24T23:53:42.530 INFO:journalctl@ceph.mon.c.smithi188.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout: format_version: 5 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout: enable_index_compression: 1 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_align: 0 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout: max_auto_readahead_size: 262144 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout: prepopulate_block_cache: 0 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression: NoCompression 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:53:42.531 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.num_levels: 7 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:53:42.532 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:53:42.533 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:53:42.534 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:53:42.535 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.ttl: 2592000 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:53:42.536 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9927dd7c-ce3a-4843-89d8-aab8de0e9244 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714002822218124, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714002822218571, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714002822, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9927dd7c-ce3a-4843-89d8-aab8de0e9244", "db_session_id": "63YF02C3NMMMICXQAFRH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714002822218650, "job": 1, "event": "recovery_finished"} 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T23:53:42.537 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556a6110e000 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: DB pointer 0x556a610f8000 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** DB Stats ** 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:53:42.538 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** Compaction Stats [default] ** 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** Compaction Stats [default] ** 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-04-24T23:53:42.539 INFO:journalctl@ceph.mon.c.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: Block cache BinnedLRUCache@0x556a6101d090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-04-24T23:53:42.540 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: using public_addr v2:172.21.15.188:0/0 -> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: starting mon.c rank -1 at public addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] at bind addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(???) e0 preinit fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).mds e1 new map 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout: e1 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout: legacy client fscid: -1 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout: No filesystems configured 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T23:53:42.541 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.542 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Updating smithi173:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Deploying cephadm binary to smithi188 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Updating smithi173:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.543 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Added host smithi188 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3530717928' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3530717928' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:53:42.544 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Updating smithi188:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:53:42.545 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Updating smithi188:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi110:172.21.15.110=a;smithi173:172.21.15.173=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Saving service mon spec with placement smithi110:172.21.15.110=a;smithi173:172.21.15.173=b;smithi188:172.21.15.188=c;count:3 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: Deploying daemon mon.c on smithi188 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/1135174452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:53:42.546 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:42.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 ceph-mon[33285]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T23:53:42.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 bash[33252]: c462243bdd476cc2be0ef2813924cdfc06b015b89f38503519b70cab25c80685 2024-04-24T23:53:42.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:42 smithi188 systemd[1]: Started Ceph mon.c for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:53:43.933 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:53:43.934 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:53:44.179 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: mon.a calling monitor election 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: mon.c calling monitor election 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: monmap epoch 2 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: last_changed 2024-04-24T23:53:42.241281+0000 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: created 2024-04-24T23:51:50.484996+0000 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: min_mon_release 19 (squid) 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: election_strategy: 1 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: fsmap 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: mgrmap e13: x(active, since 52s) 2024-04-24T23:53:47.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: overall HEALTH_OK 2024-04-24T23:53:47.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:47.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: mon.a calling monitor election 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: mon.c calling monitor election 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: monmap epoch 2 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: last_changed 2024-04-24T23:53:42.241281+0000 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: created 2024-04-24T23:51:50.484996+0000 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: min_mon_release 19 (squid) 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: election_strategy: 1 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: fsmap 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:53:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: mgrmap e13: x(active, since 52s) 2024-04-24T23:53:47.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: overall HEALTH_OK 2024-04-24T23:53:47.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:47.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:48.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.673+0000 7f9401c20700 1 -- 172.21.15.188:0/638078366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e80035c0 msgr2=0x7f93e8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:48.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.673+0000 7f9401c20700 1 --2- 172.21.15.188:0/638078366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e80035c0 0x7f93e8005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f93fc100b50 tx=0x7f93ec00feb0 comp rx=0 tx=0).stop 2024-04-24T23:53:48.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.673+0000 7f9401c20700 1 -- 172.21.15.188:0/638078366 shutdown_connections 2024-04-24T23:53:48.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.673+0000 7f9401c20700 1 --2- 172.21.15.188:0/638078366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93e80035c0 0x7f93e8005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:48.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.673+0000 7f9401c20700 1 -- 172.21.15.188:0/638078366 >> 172.21.15.188:0/638078366 conn(0x7f93fc0fc710 msgr2=0x7f93fc0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:48.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.673+0000 7f9401c20700 1 -- 172.21.15.188:0/638078366 shutdown_connections 2024-04-24T23:53:48.674 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.673+0000 7f9401c20700 1 -- 172.21.15.188:0/638078366 wait complete. 2024-04-24T23:53:48.674 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:48 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:48.674 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:48 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:53:48.674 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:48 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:48.674 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:48 smithi188 ceph-mon[33285]: Deploying daemon mon.b on smithi173 2024-04-24T23:53:48.674 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:48 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:48.676 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.674+0000 7f9401c20700 1 Processor -- start 2024-04-24T23:53:48.676 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.674+0000 7f9401c20700 1 -- start start 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f9401c20700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 0x7f93fc1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f9401c20700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc1968f0 0x7f93fc196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f9401c20700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc108b60 con 0x7f93fc1968f0 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f9401c20700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1089e0 con 0x7f93e80035c0 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 0x7f93fc1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93f3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc1968f0 0x7f93fc196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 0x7f93fc1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53758/0 (socket says 172.21.15.188:53758) 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93fb7fe700 1 -- 172.21.15.188:0/3374781318 learned_addr learned my addr 172.21.15.188:0/3374781318 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93fb7fe700 1 -- 172.21.15.188:0/3374781318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 msgr2=0x7f93fc1963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93fb7fe700 1 -- 172.21.15.188:0/3374781318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 msgr2=0x7f93fc1963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93fb7fe700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 0x7f93fc1963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93fb7fe700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 0x7f93fc1963b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:53:48.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93f3fff700 1 -- 172.21.15.188:0/3374781318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 msgr2=0x7f93fc1963b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93f3fff700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 0x7f93fc1963b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93f3fff700 1 -- 172.21.15.188:0/3374781318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc19b400 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.675+0000 7f93f3fff700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc1968f0 0x7f93fc196d70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b4d0 tx=0x7f93e400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.676+0000 7f93f97fa700 1 -- 172.21.15.188:0/3374781318 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f93e4011630 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.676+0000 7f93f97fa700 1 -- 172.21.15.188:0/3374781318 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e4011c80 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.676+0000 7f93f97fa700 1 -- 172.21.15.188:0/3374781318 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f93e4010d90 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.676+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc19b6f0 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.676+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc19bc80 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.677+0000 7f93f97fa700 1 -- 172.21.15.188:0/3374781318 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f93e4018020 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.677+0000 7f93f97fa700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93dc038e90 0x7f93dc03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.677+0000 7f93f97fa700 1 -- 172.21.15.188:0/3374781318 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f93e404bc30 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.677+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93fc102360 con 0x7f93fc1968f0 2024-04-24T23:53:48.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.677+0000 7f93fb7fe700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93dc038e90 0x7f93dc03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:48.679 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.677+0000 7f93fb7fe700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93dc038e90 0x7f93dc03b350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00fa60 tx=0x7f93ec005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:48.682 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.683+0000 7f93f97fa700 1 -- 172.21.15.188:0/3374781318 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f93e401f020 con 0x7f93fc1968f0 2024-04-24T23:53:48.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:48 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:48.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:48 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:53:48.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:48 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:48.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:48 smithi110 ceph-mon[29187]: Deploying daemon mon.b on smithi173 2024-04-24T23:53:48.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:48 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:48.963 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.963+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f93fc19c800 con 0x7f93fc1968f0 2024-04-24T23:53:48.963 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.964+0000 7f93f97fa700 1 -- 172.21.15.188:0/3374781318 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f93e4018570 con 0x7f93fc1968f0 2024-04-24T23:53:48.964 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:53:48.964 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":2,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","modified":"2024-04-24T23:53:42.241281Z","created":"2024-04-24T23:51:50.484996Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93dc038e90 msgr2=0x7f93dc03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93dc038e90 0x7f93dc03b350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00fa60 tx=0x7f93ec005a90 comp rx=0 tx=0).stop 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc1968f0 msgr2=0x7f93fc196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc1968f0 0x7f93fc196d70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b4d0 tx=0x7f93e400b890 comp rx=0 tx=0).stop 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 shutdown_connections 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93dc038e90 0x7f93dc03b350 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e80035c0 0x7f93fc1963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 --2- 172.21.15.188:0/3374781318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93fc1968f0 0x7f93fc196d70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 >> 172.21.15.188:0/3374781318 conn(0x7f93fc0fc710 msgr2=0x7f93fc0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:48.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 shutdown_connections 2024-04-24T23:53:48.967 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:48.966+0000 7f9401c20700 1 -- 172.21.15.188:0/3374781318 wait complete. 2024-04-24T23:53:48.967 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 2 2024-04-24T23:53:49.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:49 smithi188 ceph-mon[33285]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:49.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:49 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/3374781318' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:53:49.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:49 smithi110 ceph-mon[29187]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:49.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:49 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/3374781318' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:53:50.504 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:53:50.504 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:53:50.746 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:53:51.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:51 smithi173 systemd[1]: Started Ceph mon.b for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:53:56.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.582+0000 7f02610ae700 1 -- 172.21.15.188:0/2118610475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02440035c0 msgr2=0x7f0244005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:56.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.582+0000 7f02610ae700 1 --2- 172.21.15.188:0/2118610475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02440035c0 0x7f0244005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f025c100b50 tx=0x7f025000fe60 comp rx=0 tx=0).stop 2024-04-24T23:53:56.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.582+0000 7f02610ae700 1 -- 172.21.15.188:0/2118610475 shutdown_connections 2024-04-24T23:53:56.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.582+0000 7f02610ae700 1 --2- 172.21.15.188:0/2118610475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02440035c0 0x7f0244005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:56.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.582+0000 7f02610ae700 1 -- 172.21.15.188:0/2118610475 >> 172.21.15.188:0/2118610475 conn(0x7f025c0fc710 msgr2=0x7f025c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:56.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.582+0000 7f02610ae700 1 -- 172.21.15.188:0/2118610475 shutdown_connections 2024-04-24T23:53:56.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.582+0000 7f02610ae700 1 -- 172.21.15.188:0/2118610475 wait complete. 2024-04-24T23:53:56.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.583+0000 7f02610ae700 1 Processor -- start 2024-04-24T23:53:56.585 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f02610ae700 1 -- start start 2024-04-24T23:53:56.585 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f02610ae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 0x7f025c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:56.585 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f02610ae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 0x7f025c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:56.585 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f02610ae700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c1a1820 0x7f025c1a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:56.585 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f02610ae700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f025c108ac0 con 0x7f025c1a1820 2024-04-24T23:53:56.585 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f02610ae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f025c108940 con 0x7f02440035c0 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f02610ae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f025c108c40 con 0x7f025c198cf0 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025a59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 0x7f025c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025a59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 0x7f025c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:45780/0 (socket says 172.21.15.188:45780) 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025a59c700 1 -- 172.21.15.188:0/550805604 learned_addr learned my addr 172.21.15.188:0/550805604 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025ad9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 0x7f025c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025b59e700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c1a1820 0x7f025c1a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025ad9d700 1 -- 172.21.15.188:0/550805604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 msgr2=0x7f025c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025ad9d700 1 -- 172.21.15.188:0/550805604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 msgr2=0x7f025c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025ad9d700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 0x7f025c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025a59c700 1 -- 172.21.15.188:0/550805604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 msgr2=0x7f025c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025ad9d700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 0x7f025c1987b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:53:56.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025a59c700 1 -- 172.21.15.188:0/550805604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 msgr2=0x7f025c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025a59c700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 0x7f025c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.584+0000 7f025a59c700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 0x7f025c199170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f025b59e700 1 -- 172.21.15.188:0/550805604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 msgr2=0x7f025c199170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f025b59e700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 0x7f025c199170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f025b59e700 1 -- 172.21.15.188:0/550805604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 msgr2=0x7f025c1987b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f025b59e700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 0x7f025c1987b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f025b59e700 1 -- 172.21.15.188:0/550805604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c1a43d0 con 0x7f025c1a1820 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f025b59e700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c1a1820 0x7f025c1a3c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f025400bd50 tx=0x7f025400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f0243fff700 1 -- 172.21.15.188:0/550805604 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254019070 con 0x7f025c1a1820 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f0243fff700 1 -- 172.21.15.188:0/550805604 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f025400cd10 con 0x7f025c1a1820 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c1a46c0 con 0x7f025c1a1820 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1a4c00 con 0x7f025c1a1820 2024-04-24T23:53:56.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.585+0000 7f0243fff700 1 -- 172.21.15.188:0/550805604 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254012700 con 0x7f025c1a1820 2024-04-24T23:53:56.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.586+0000 7f0243fff700 1 -- 172.21.15.188:0/550805604 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f025401c070 con 0x7f025c1a1820 2024-04-24T23:53:56.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.586+0000 7f0243fff700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0248039580 0x7f024803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:56.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.586+0000 7f0243fff700 1 -- 172.21.15.188:0/550805604 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f025404ca90 con 0x7f025c1a1820 2024-04-24T23:53:56.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.586+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c063890 con 0x7f025c1a1820 2024-04-24T23:53:56.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.586+0000 7f025ad9d700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0248039580 0x7f024803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:56.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.587+0000 7f025ad9d700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0248039580 0x7f024803ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f025000fa10 tx=0x7f0250005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:56.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.592+0000 7f0243fff700 1 -- 172.21.15.188:0/550805604 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0254013c30 con 0x7f025c1a1820 2024-04-24T23:53:56.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: mon.a calling monitor election 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: mon.c calling monitor election 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: mon.b calling monitor election 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:53:56.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: monmap epoch 3 2024-04-24T23:53:56.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:56.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: last_changed 2024-04-24T23:53:51.291822+0000 2024-04-24T23:53:56.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: created 2024-04-24T23:51:50.484996+0000 2024-04-24T23:53:56.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: min_mon_release 19 (squid) 2024-04-24T23:53:56.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: election_strategy: 1 2024-04-24T23:53:56.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-04-24T23:53:56.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-24T23:53:56.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-24T23:53:56.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: fsmap 2024-04-24T23:53:56.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:53:56.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: mgrmap e13: x(active, since 61s) 2024-04-24T23:53:56.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: overall HEALTH_OK 2024-04-24T23:53:56.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:53:56.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: mon.a calling monitor election 2024-04-24T23:53:56.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:56.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: mon.c calling monitor election 2024-04-24T23:53:56.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:56.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: mon.b calling monitor election 2024-04-24T23:53:56.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: monmap epoch 3 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: last_changed 2024-04-24T23:53:51.291822+0000 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: created 2024-04-24T23:51:50.484996+0000 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: min_mon_release 19 (squid) 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: election_strategy: 1 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-24T23:53:56.788 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-24T23:53:56.789 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: fsmap 2024-04-24T23:53:56.789 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:53:56.789 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: mgrmap e13: x(active, since 61s) 2024-04-24T23:53:56.789 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: overall HEALTH_OK 2024-04-24T23:53:56.789 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.789 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.789 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: mon.a calling monitor election 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: mon.c calling monitor election 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: mon.b calling monitor election 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: monmap epoch 3 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: last_changed 2024-04-24T23:53:51.291822+0000 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: created 2024-04-24T23:51:50.484996+0000 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: min_mon_release 19 (squid) 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: election_strategy: 1 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: fsmap 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: mgrmap e13: x(active, since 61s) 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: overall HEALTH_OK 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:56.887 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.887+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f025c1a4f10 con 0x7f025c1a1820 2024-04-24T23:53:56.888 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.888+0000 7f0243fff700 1 -- 172.21.15.188:0/550805604 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f025401d730 con 0x7f025c1a1820 2024-04-24T23:53:56.888 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-24T23:53:56.888 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":3,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","modified":"2024-04-24T23:53:51.291822Z","created":"2024-04-24T23:51:50.484996Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0248039580 msgr2=0x7f024803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0248039580 0x7f024803ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f025000fa10 tx=0x7f0250005de0 comp rx=0 tx=0).stop 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c1a1820 msgr2=0x7f025c1a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c1a1820 0x7f025c1a3c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f025400bd50 tx=0x7f025400be30 comp rx=0 tx=0).stop 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 shutdown_connections 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0248039580 0x7f024803ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f025c1a1820 0x7f025c1a3c10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02440035c0 0x7f025c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 --2- 172.21.15.188:0/550805604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c198cf0 0x7f025c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:56.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 >> 172.21.15.188:0/550805604 conn(0x7f025c0fc710 msgr2=0x7f025c104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:56.891 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 shutdown_connections 2024-04-24T23:53:56.891 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:53:56.890+0000 7f02610ae700 1 -- 172.21.15.188:0/550805604 wait complete. 2024-04-24T23:53:56.891 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 3 2024-04-24T23:53:57.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:57.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:57 smithi188 ceph-mon[33285]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:57.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:57.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:57 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/550805604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:53:57.513 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:57.539 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T23:53:57.539 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T23:53:57.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:57.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:57 smithi173 ceph-mon[32402]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:57.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:57.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:57 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/550805604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:53:57.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:57.784 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:53:57.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:57.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:57 smithi110 ceph-mon[29187]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:57.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:53:57.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:57 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/550805604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:53:57.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:53:59.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.212 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: Updating smithi110:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: Updating smithi173:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: Updating smithi188:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.213 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: Updating smithi110:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: Updating smithi173:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: Updating smithi188:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.351 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:53:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: Updating smithi110:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: Updating smithi173:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: Updating smithi188:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:53:59.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.439+0000 7f058fe4e700 1 -- 172.21.15.110:0/2608043584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 msgr2=0x7f0588101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:59.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.439+0000 7f058fe4e700 1 --2- 172.21.15.110:0/2608043584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 0x7f0588101a70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0578009910 tx=0x7f0578009c20 comp rx=0 tx=0).stop 2024-04-24T23:53:59.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.439+0000 7f058fe4e700 1 -- 172.21.15.110:0/2608043584 shutdown_connections 2024-04-24T23:53:59.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.439+0000 7f058fe4e700 1 --2- 172.21.15.110:0/2608043584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 0x7f0588101a70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:59.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.439+0000 7f058fe4e700 1 -- 172.21.15.110:0/2608043584 >> 172.21.15.110:0/2608043584 conn(0x7f0588078130 msgr2=0x7f0588078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:59.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.439+0000 7f058fe4e700 1 -- 172.21.15.110:0/2608043584 shutdown_connections 2024-04-24T23:53:59.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.439+0000 7f058fe4e700 1 -- 172.21.15.110:0/2608043584 wait complete. 2024-04-24T23:53:59.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058fe4e700 1 Processor -- start 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058fe4e700 1 -- start start 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058fe4e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 0x7f058819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058fe4e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f058819dba0 0x7f0588197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058fe4e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0588197c70 0x7f0588198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058fe4e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f058810b0a0 con 0x7f0588101690 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058fe4e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f058810af20 con 0x7f0588197c70 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058fe4e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f058810b220 con 0x7f058819dba0 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058dbea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 0x7f058819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.440+0000 7f058dbea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 0x7f058819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38036/0 (socket says 172.21.15.110:38036) 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058dbea700 1 -- 172.21.15.110:0/2690645340 learned_addr learned my addr 172.21.15.110:0/2690645340 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058d3e9700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f058819dba0 0x7f0588197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058e3eb700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0588197c70 0x7f0588198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:59.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058dbea700 1 -- 172.21.15.110:0/2690645340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f058819dba0 msgr2=0x7f0588197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058dbea700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f058819dba0 0x7f0588197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058dbea700 1 -- 172.21.15.110:0/2690645340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0588197c70 msgr2=0x7f0588198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058dbea700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0588197c70 0x7f0588198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058dbea700 1 -- 172.21.15.110:0/2690645340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05881a4410 con 0x7f0588101690 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058dbea700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 0x7f058819d660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0578004f80 tx=0x7f0578004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f057effd700 1 -- 172.21.15.110:0/2690645340 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578026070 con 0x7f0588101690 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f057effd700 1 -- 172.21.15.110:0/2690645340 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f057800b6f0 con 0x7f0588101690 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05881a46a0 con 0x7f0588101690 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f057effd700 1 -- 172.21.15.110:0/2690645340 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578009e60 con 0x7f0588101690 2024-04-24T23:53:59.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.441+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05881a4c30 con 0x7f0588101690 2024-04-24T23:53:59.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.442+0000 7f057effd700 1 -- 172.21.15.110:0/2690645340 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f0578029070 con 0x7f0588101690 2024-04-24T23:53:59.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.442+0000 7f057effd700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0574039580 0x7f057403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:53:59.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.442+0000 7f057effd700 1 -- 172.21.15.110:0/2690645340 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f057804cbf0 con 0x7f0588101690 2024-04-24T23:53:59.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.442+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0588102440 con 0x7f0588101690 2024-04-24T23:53:59.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.443+0000 7f058d3e9700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0574039580 0x7f057403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:53:59.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.443+0000 7f058d3e9700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0574039580 0x7f057403ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0584009ad0 tx=0x7f0584009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:53:59.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.452+0000 7f057effd700 1 -- 172.21.15.110:0/2690645340 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0578030020 con 0x7f0588101690 2024-04-24T23:53:59.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.661+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f058802db30 con 0x7f0588101690 2024-04-24T23:53:59.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.661+0000 7f057effd700 1 -- 172.21.15.110:0/2690645340 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f0578013070 con 0x7f0588101690 2024-04-24T23:53:59.661 INFO:teuthology.orchestra.run.smithi110.stdout:# minimal ceph.conf for 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:59.661 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2024-04-24T23:53:59.661 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-24T23:53:59.661 INFO:teuthology.orchestra.run.smithi110.stdout: mon_host = [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-04-24T23:53:59.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0574039580 msgr2=0x7f057403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:59.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0574039580 0x7f057403ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0584009ad0 tx=0x7f0584009210 comp rx=0 tx=0).stop 2024-04-24T23:53:59.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 msgr2=0x7f058819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 0x7f058819d660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0578004f80 tx=0x7f0578004fb0 comp rx=0 tx=0).stop 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 shutdown_connections 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0574039580 0x7f057403ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0588197c70 0x7f0588198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0588101690 0x7f058819d660 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 --2- 172.21.15.110:0/2690645340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f058819dba0 0x7f0588197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 >> 172.21.15.110:0/2690645340 conn(0x7f0588078130 msgr2=0x7f0588108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.663+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 shutdown_connections 2024-04-24T23:53:59.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:53:59.664+0000 7f058fe4e700 1 -- 172.21.15.110:0/2690645340 wait complete. 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: Reconfiguring daemon mon.a on smithi110 2024-04-24T23:54:00.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:53:59 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2690645340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:00.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:54:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: Reconfiguring daemon mon.a on smithi110 2024-04-24T23:54:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:53:59 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2690645340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:00.351 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.351 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.351 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:54:00.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:00.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:00.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:00.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: Reconfiguring daemon mon.a on smithi110 2024-04-24T23:54:00.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:00.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:53:59 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2690645340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:00.691 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T23:54:00.691 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:54:00.691 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:54:00.720 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:54:00.721 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:54:00.785 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:54:00.785 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:54:00.817 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:54:00.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:54:00.883 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:54:00.883 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:54:00.914 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:54:00.914 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:54:00.981 INFO:tasks.cephadm:Adding mgr.x on smithi110 2024-04-24T23:54:00.982 INFO:tasks.cephadm:Adding mgr.y on smithi173 2024-04-24T23:54:00.982 INFO:tasks.cephadm:Adding mgr.z on smithi188 2024-04-24T23:54:00.982 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi110=x;smithi173=y;smithi188=z' 2024-04-24T23:54:01.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:00 smithi173 ceph-mon[32402]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:01.251 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:54:01.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:00 smithi188 ceph-mon[33285]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:01.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:00 smithi110 ceph-mon[29187]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:02.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:02.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:02.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:02 smithi173 ceph-mon[32402]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:54:02.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:02.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:02.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:02.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:02 smithi173 ceph-mon[32402]: Reconfiguring daemon mon.b on smithi173 2024-04-24T23:54:02.509 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:02.509 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:02.509 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:02 smithi188 ceph-mon[33285]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:54:02.509 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:02.509 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:02.509 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:02.509 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:02 smithi188 ceph-mon[33285]: Reconfiguring daemon mon.b on smithi173 2024-04-24T23:54:02.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:02.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:02.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:02 smithi110 ceph-mon[29187]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:54:02.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:02.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:02.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:02.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:02 smithi110 ceph-mon[29187]: Reconfiguring daemon mon.b on smithi173 2024-04-24T23:54:02.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.729+0000 7f8f11f35700 1 -- 172.21.15.188:0/4063070358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c100610 msgr2=0x7f8f0c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.729+0000 7f8f11f35700 1 --2- 172.21.15.188:0/4063070358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c100610 0x7f8f0c100a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009910 tx=0x7f8f00009c20 comp rx=0 tx=0).stop 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.729+0000 7f8f11f35700 1 -- 172.21.15.188:0/4063070358 shutdown_connections 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.729+0000 7f8f11f35700 1 --2- 172.21.15.188:0/4063070358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c100610 0x7f8f0c100a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.729+0000 7f8f11f35700 1 -- 172.21.15.188:0/4063070358 >> 172.21.15.188:0/4063070358 conn(0x7f8f0c0fc230 msgr2=0x7f8f0c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.729+0000 7f8f11f35700 1 -- 172.21.15.188:0/4063070358 shutdown_connections 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.729+0000 7f8f11f35700 1 -- 172.21.15.188:0/4063070358 wait complete. 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.730+0000 7f8f11f35700 1 Processor -- start 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.730+0000 7f8f11f35700 1 -- start start 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.730+0000 7f8f11f35700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f0c100610 0x7f8f0c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.730+0000 7f8f11f35700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f0c199c20 0x7f8f0c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.730+0000 7f8f11f35700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c19a0a0 0x7f8f0c1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:02.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.730+0000 7f8f11f35700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c109ec0 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.730+0000 7f8f11f35700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c109d40 con 0x7f8f0c199c20 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.730+0000 7f8f11f35700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c10a040 con 0x7f8f0c100610 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f0c100610 0x7f8f0c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c19a0a0 0x7f8f0c1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f0c100610 0x7f8f0c1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:32858/0 (socket says 172.21.15.188:32858) 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c19a0a0 0x7f8f0c1a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.188:59540/0 (socket says 172.21.15.188:59540) 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 -- 172.21.15.188:0/3648832962 learned_addr learned my addr 172.21.15.188:0/3648832962 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0affd700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f0c199c20 0x7f8f0c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 -- 172.21.15.188:0/3648832962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f0c100610 msgr2=0x7f8f0c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f0c100610 0x7f8f0c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 -- 172.21.15.188:0/3648832962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f0c199c20 msgr2=0x7f8f0c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f0c199c20 0x7f8f0c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 -- 172.21.15.188:0/3648832962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c1a1af0 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f0bfff700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c19a0a0 0x7f8f0c1a1220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400ba50 tx=0x7f8f0400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f08ff9700 1 -- 172.21.15.188:0/3648832962 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04019070 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f08ff9700 1 -- 172.21.15.188:0/3648832962 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f0400cde0 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c1a1de0 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.731+0000 7f8f08ff9700 1 -- 172.21.15.188:0/3648832962 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04007350 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.732+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c1a2360 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.732+0000 7f8f08ff9700 1 -- 172.21.15.188:0/3648832962 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8f0401c070 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.732+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0c063890 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.733+0000 7f8f08ff9700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ef4039580 0x7f8ef403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.733+0000 7f8f08ff9700 1 -- 172.21.15.188:0/3648832962 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f0404b880 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.733+0000 7f8f0b7fe700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ef4039580 0x7f8ef403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:02.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.733+0000 7f8f0b7fe700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ef4039580 0x7f8ef403ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f00005190 tx=0x7f8f0001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:54:02.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.738+0000 7f8f08ff9700 1 -- 172.21.15.188:0/3648832962 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f0401b2c0 con 0x7f8f0c19a0a0 2024-04-24T23:54:02.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.954+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi110=x;smithi173=y;smithi188=z", "target": ["mon-mgr", ""]}) -- 0x7f8f0c062700 con 0x7f8ef4039580 2024-04-24T23:54:02.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.961+0000 7f8f08ff9700 1 -- 172.21.15.188:0/3648832962 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8f0c062700 con 0x7f8ef4039580 2024-04-24T23:54:02.961 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mgr update... 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ef4039580 msgr2=0x7f8ef403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ef4039580 0x7f8ef403ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f00005190 tx=0x7f8f0001a040 comp rx=0 tx=0).stop 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c19a0a0 msgr2=0x7f8f0c1a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c19a0a0 0x7f8f0c1a1220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400ba50 tx=0x7f8f0400bd60 comp rx=0 tx=0).stop 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 shutdown_connections 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ef4039580 0x7f8ef403ba40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f0c19a0a0 0x7f8f0c1a1220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f0c100610 0x7f8f0c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 --2- 172.21.15.188:0/3648832962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f0c199c20 0x7f8f0c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 >> 172.21.15.188:0/3648832962 conn(0x7f8f0c0fc230 msgr2=0x7f8f0c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 shutdown_connections 2024-04-24T23:54:02.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:02.964+0000 7f8f11f35700 1 -- 172.21.15.188:0/3648832962 wait complete. 2024-04-24T23:54:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:03 smithi188 ceph-mon[33285]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:03 smithi110 ceph-mon[29187]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:03.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:03.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:03.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:03.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.557 DEBUG:teuthology.orchestra.run.smithi173:mgr.y> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T23:54:03.560 DEBUG:teuthology.orchestra.run.smithi188:mgr.z> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T23:54:03.562 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T23:54:03.562 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-24T23:54:03.562 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:54:03.581 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:54:03.581 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-04-24T23:54:03.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:03 smithi173 ceph-mon[32402]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:03.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:54:03.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:54:03.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:03.583 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-04-24 23:53:07.073612269 +0000 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-24 23:48:38.186122752 +0000 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-24 23:48:38.186122752 +0000 2024-04-24T23:54:03.640 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-04-24 23:48:38.186122752 +0000 2024-04-24T23:54:03.641 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:54:03.708 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-04-24T23:54:03.708 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-04-24T23:54:03.708 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000108296 s, 4.7 MB/s 2024-04-24T23:54:03.710 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:54:03.769 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-04-24T23:54:03.826 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:54:03.826 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:03.826 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-24T23:54:03.826 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:03.826 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:03.826 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-04-24 23:53:07.119611476 +0000 2024-04-24T23:54:03.827 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-24 23:48:38.518116961 +0000 2024-04-24T23:54:03.827 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-24 23:48:38.518116961 +0000 2024-04-24T23:54:03.827 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-04-24 23:48:38.518116961 +0000 2024-04-24T23:54:03.827 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:54:03.891 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-04-24T23:54:03.892 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-04-24T23:54:03.892 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000162585 s, 3.1 MB/s 2024-04-24T23:54:03.893 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:54:03.950 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-04-24 23:53:07.163610717 +0000 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-24 23:48:38.847111221 +0000 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-24 23:48:38.847111221 +0000 2024-04-24T23:54:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-04-24 23:48:38.847111221 +0000 2024-04-24T23:54:04.011 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:54:04.075 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-04-24T23:54:04.075 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-04-24T23:54:04.075 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000124491 s, 4.1 MB/s 2024-04-24T23:54:04.077 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:54:04.134 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-04-24 23:53:07.199610096 +0000 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-24 23:48:39.179105430 +0000 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-24 23:48:39.179105430 +0000 2024-04-24T23:54:04.190 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-04-24 23:48:39.179105430 +0000 2024-04-24T23:54:04.191 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:54:04.254 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-04-24T23:54:04.255 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-04-24T23:54:04.255 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000117118 s, 4.4 MB/s 2024-04-24T23:54:04.256 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:54:04.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:04 smithi188 ceph-mon[33285]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:54:04.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:04 smithi188 ceph-mon[33285]: Reconfiguring daemon mon.c on smithi188 2024-04-24T23:54:04.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:04 smithi188 ceph-mon[33285]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi110=x;smithi173=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:04.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:04 smithi188 ceph-mon[33285]: Saving service mgr spec with placement smithi110=x;smithi173=y;smithi188=z;count:3 2024-04-24T23:54:04.312 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-24T23:54:04.312 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:54:04.331 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:54:04.332 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-04-24T23:54:04.390 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:54:04.390 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:04.390 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-24T23:54:04.390 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:04.390 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:04.390 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-24 23:53:50.748069597 +0000 2024-04-24T23:54:04.390 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-24 23:48:39.634856048 +0000 2024-04-24T23:54:04.391 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-24 23:48:39.634856048 +0000 2024-04-24T23:54:04.391 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-24 23:48:39.634856048 +0000 2024-04-24T23:54:04.391 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:54:04.455 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:04 smithi173 ceph-mon[32402]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:54:04.455 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:04 smithi173 ceph-mon[32402]: Reconfiguring daemon mon.c on smithi188 2024-04-24T23:54:04.455 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:04 smithi173 ceph-mon[32402]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi110=x;smithi173=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:04.455 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:04 smithi173 ceph-mon[32402]: Saving service mgr spec with placement smithi110=x;smithi173=y;smithi188=z;count:3 2024-04-24T23:54:04.457 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-24T23:54:04.457 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-24T23:54:04.457 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.00028702 s, 1.8 MB/s 2024-04-24T23:54:04.459 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:54:04.518 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-04-24T23:54:04.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:04 smithi110 ceph-mon[29187]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:54:04.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:04 smithi110 ceph-mon[29187]: Reconfiguring daemon mon.c on smithi188 2024-04-24T23:54:04.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:04 smithi110 ceph-mon[29187]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi110=x;smithi173=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:04.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:04 smithi110 ceph-mon[29187]: Saving service mgr spec with placement smithi110=x;smithi173=y;smithi188=z;count:3 2024-04-24T23:54:04.575 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:54:04.575 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:04.575 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-24T23:54:04.575 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:04.575 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:04.575 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-24 23:53:50.749069574 +0000 2024-04-24T23:54:04.576 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-24 23:48:39.980848243 +0000 2024-04-24T23:54:04.576 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-24 23:48:39.980848243 +0000 2024-04-24T23:54:04.576 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-24 23:48:39.980848243 +0000 2024-04-24T23:54:04.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:54:04.642 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-24T23:54:04.642 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-24T23:54:04.642 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000298969 s, 1.7 MB/s 2024-04-24T23:54:04.644 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:54:04.701 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-04-24T23:54:04.757 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:54:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:04.758 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T23:54:04.758 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:04.758 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:04.758 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-24 23:53:50.749069574 +0000 2024-04-24T23:54:04.758 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-24 23:48:40.320840573 +0000 2024-04-24T23:54:04.758 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-24 23:48:40.320840573 +0000 2024-04-24T23:54:04.758 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-24 23:48:40.320840573 +0000 2024-04-24T23:54:04.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:54:04.822 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-24T23:54:04.822 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-24T23:54:04.823 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000190734 s, 2.7 MB/s 2024-04-24T23:54:04.824 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:54:04.881 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-04-24T23:54:04.937 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:54:04.937 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:04.937 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T23:54:04.937 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:04.938 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:04.938 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-24 23:53:50.750069552 +0000 2024-04-24T23:54:04.938 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-24 23:48:40.669832700 +0000 2024-04-24T23:54:04.938 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-24 23:48:40.669832700 +0000 2024-04-24T23:54:04.938 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-24 23:48:40.669832700 +0000 2024-04-24T23:54:04.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:54:05.000 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-24T23:54:05.000 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-24T23:54:05.001 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000194895 s, 2.6 MB/s 2024-04-24T23:54:05.002 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:54:05.058 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-24T23:54:05.058 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:54:05.077 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:54:05.077 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-24 23:53:40.758084117 +0000 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-24 23:48:39.384913680 +0000 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-24 23:48:39.384913680 +0000 2024-04-24T23:54:05.134 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-24 23:48:39.384913680 +0000 2024-04-24T23:54:05.134 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:54:05.198 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-24T23:54:05.198 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-24T23:54:05.198 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000198257 s, 2.6 MB/s 2024-04-24T23:54:05.199 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:54:05.256 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-04-24T23:54:05.311 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:54:05.312 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:05.312 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T23:54:05.312 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:05.312 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:05.312 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-24 23:53:40.759084116 +0000 2024-04-24T23:54:05.312 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-24 23:48:39.722913387 +0000 2024-04-24T23:54:05.312 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-24 23:48:39.722913387 +0000 2024-04-24T23:54:05.312 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-24 23:48:39.722913387 +0000 2024-04-24T23:54:05.312 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:54:05.374 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-24T23:54:05.375 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-24T23:54:05.375 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000219618 s, 2.3 MB/s 2024-04-24T23:54:05.376 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:54:05.432 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-24 23:53:40.759084116 +0000 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-24 23:48:40.065913090 +0000 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-24 23:48:40.065913090 +0000 2024-04-24T23:54:05.487 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-24 23:48:40.065913090 +0000 2024-04-24T23:54:05.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:54:05.550 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-24T23:54:05.550 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-24T23:54:05.550 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.00019533 s, 2.6 MB/s 2024-04-24T23:54:05.551 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:54:05.606 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-04-24T23:54:05.663 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:54:05.663 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:54:05.663 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-24T23:54:05.663 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:54:05.663 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:54:05.663 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-24 23:53:40.759084116 +0000 2024-04-24T23:54:05.663 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-24 23:48:40.400912799 +0000 2024-04-24T23:54:05.663 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-24 23:48:40.400912799 +0000 2024-04-24T23:54:05.664 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-24 23:48:40.400912799 +0000 2024-04-24T23:54:05.664 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:54:05.727 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-24T23:54:05.727 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-24T23:54:05.728 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000186043 s, 2.8 MB/s 2024-04-24T23:54:05.729 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:54:05.785 INFO:tasks.cephadm:Deploying osd.0 on smithi110 with /dev/vg_nvme/lv_4... 2024-04-24T23:54:05.785 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:54:05.886 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:05.886 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:05.886 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:05.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:05.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:05.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:05.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:05.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:05.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:54:05.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:05.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:06.013 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:54:06.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:06.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:06.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:06.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:06 smithi110 ceph-mon[29187]: Deploying daemon mgr.z on smithi188 2024-04-24T23:54:06.889 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:06 smithi188 ceph-mon[33285]: Deploying daemon mgr.z on smithi188 2024-04-24T23:54:07.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:06 smithi173 ceph-mon[32402]: Deploying daemon mgr.z on smithi188 2024-04-24T23:54:07.246 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 systemd[1]: Starting Ceph mgr.z for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:54:07.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 podman[35132]: 2024-04-24 23:54:07.453539415 +0000 UTC m=+0.105015758 container create 7ac6894c5fa2d77003235b052ca6bb0b698fd8428179132f3b4ce1d643d90469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-04-24T23:54:07.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 podman[35132]: 2024-04-24 23:54:07.366788803 +0000 UTC m=+0.018265115 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:54:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:54:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:54:07.914 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-04-24T23:54:07.914 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-04-24T23:54:07.914 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00844447 s, 1.2 GB/s 2024-04-24T23:54:07.915 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-04-24T23:54:07.915 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:07 smithi110 ceph-mon[29187]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:07.984 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:07 smithi188 ceph-mon[33285]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:07.984 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 podman[35132]: 2024-04-24 23:54:07.647332901 +0000 UTC m=+0.298809198 container init 7ac6894c5fa2d77003235b052ca6bb0b698fd8428179132f3b4ce1d643d90469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T23:54:07.984 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 podman[35132]: 2024-04-24 23:54:07.650414858 +0000 UTC m=+0.301891162 container start 7ac6894c5fa2d77003235b052ca6bb0b698fd8428179132f3b4ce1d643d90469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T23:54:07.984 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 ceph-mgr[35151]: -- 172.21.15.188:0/1999680836 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e866305a0 con 0x557e86605000 2024-04-24T23:54:07.985 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 bash[35132]: 7ac6894c5fa2d77003235b052ca6bb0b698fd8428179132f3b4ce1d643d90469 2024-04-24T23:54:07.985 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 systemd[1]: Started Ceph mgr.z for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:54:07.985 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:07.804+0000 7fc66f125200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:54:07.985 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:07 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:07.984+0000 7fc66f125200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:54:08.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:07 smithi173 ceph-mon[32402]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:08.266 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:08.086+0000 7fc66f125200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:54:08.733 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-04-24T23:54:08.766 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:08.394+0000 7fc66f125200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:54:08.767 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:08.477+0000 7fc66f125200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:54:08.974 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:54:09.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:54:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: Deploying daemon mgr.y on smithi173 2024-04-24T23:54:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:08 smithi110 ceph-mon[29187]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:09.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:09.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:54:09.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:09.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:09.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: Deploying daemon mgr.y on smithi173 2024-04-24T23:54:09.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:08 smithi173 ceph-mon[32402]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:54:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:09.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:09.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: Deploying daemon mgr.y on smithi173 2024-04-24T23:54:09.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:08 smithi188 ceph-mon[33285]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:09.851 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:09 smithi173 systemd[1]: Starting Ceph mgr.y for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:54:09.996 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:09 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:09.665+0000 7fc66f125200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:54:09.996 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:09 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:09.830+0000 7fc66f125200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:54:10.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:10.000+0000 7fc66f125200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:54:10.269 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:09 smithi173 podman[33777]: 2024-04-24 23:54:09.953947917 +0000 UTC m=+0.102636569 container create ab3af38ff9f5b9f1126491f0567098657b46e1c3345896e87cfa2b274074c125 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-24T23:54:10.269 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:09 smithi173 podman[33777]: 2024-04-24 23:54:09.869232268 +0000 UTC m=+0.017920920 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:54:10.270 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 podman[33777]: 2024-04-24 23:54:10.114474026 +0000 UTC m=+0.263162679 container init ab3af38ff9f5b9f1126491f0567098657b46e1c3345896e87cfa2b274074c125 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T23:54:10.270 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 podman[33777]: 2024-04-24 23:54:10.116843205 +0000 UTC m=+0.265531859 container start ab3af38ff9f5b9f1126491f0567098657b46e1c3345896e87cfa2b274074c125 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T23:54:10.270 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 ceph-mgr[33796]: -- 172.21.15.173:0/718130549 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562778bee5a0 con 0x562778bc3000 2024-04-24T23:54:10.270 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 bash[33777]: ab3af38ff9f5b9f1126491f0567098657b46e1c3345896e87cfa2b274074c125 2024-04-24T23:54:10.270 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 systemd[1]: Started Ceph mgr.y for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:54:10.270 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:10.269+0000 7ff59c831200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:54:10.565 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:10.457+0000 7ff59c831200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:54:10.565 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:10.565+0000 7ff59c831200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:54:10.587 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:10.284+0000 7fc66f125200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:54:10.587 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:10.374+0000 7fc66f125200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:54:10.587 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:10.587+0000 7fc66f125200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:54:10.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.626+0000 7fd4b8965700 1 -- 172.21.15.110:0/2430031739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b4100d20 msgr2=0x7fd4b410a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:10.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.626+0000 7fd4b8965700 1 --2- 172.21.15.110:0/2430031739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b4100d20 0x7fd4b410a8d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00b0d0 tx=0x7fd4ac00b3e0 comp rx=0 tx=0).stop 2024-04-24T23:54:10.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.626+0000 7fd4b8965700 1 -- 172.21.15.110:0/2430031739 shutdown_connections 2024-04-24T23:54:10.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.626+0000 7fd4b8965700 1 --2- 172.21.15.110:0/2430031739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b4100d20 0x7fd4b410a8d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:10.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.626+0000 7fd4b8965700 1 --2- 172.21.15.110:0/2430031739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4b41038e0 0x7fd4b4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:10.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.626+0000 7fd4b8965700 1 --2- 172.21.15.110:0/2430031739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4b4100360 0x7fd4b41007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:10.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.626+0000 7fd4b8965700 1 -- 172.21.15.110:0/2430031739 >> 172.21.15.110:0/2430031739 conn(0x7fd4b40fbf00 msgr2=0x7fd4b40fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:54:10.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.627+0000 7fd4b8965700 1 -- 172.21.15.110:0/2430031739 shutdown_connections 2024-04-24T23:54:10.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.627+0000 7fd4b8965700 1 -- 172.21.15.110:0/2430031739 wait complete. 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.627+0000 7fd4b8965700 1 Processor -- start 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.627+0000 7fd4b8965700 1 -- start start 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.627+0000 7fd4b8965700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4b4100360 0x7fd4b4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b8965700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4b4100d20 0x7fd4b41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b8965700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b41038e0 0x7fd4b419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b8965700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b410ddf0 con 0x7fd4b41038e0 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b8965700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b410dc70 con 0x7fd4b4100d20 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b8965700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b410df70 con 0x7fd4b4100360 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b41038e0 0x7fd4b419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:10.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4b4100360 0x7fd4b4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b41038e0 0x7fd4b419f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45012/0 (socket says 172.21.15.110:45012) 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b259c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4b4100360 0x7fd4b4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48714/0 (socket says 172.21.15.110:48714) 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b1d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4b4100d20 0x7fd4b41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 -- 172.21.15.110:0/3869796963 learned_addr learned my addr 172.21.15.110:0/3869796963 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 -- 172.21.15.110:0/3869796963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4b4100360 msgr2=0x7fd4b4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4b4100360 0x7fd4b4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 -- 172.21.15.110:0/3869796963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4b4100d20 msgr2=0x7fd4b41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4b4100d20 0x7fd4b41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 -- 172.21.15.110:0/3869796963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b419ffe0 con 0x7fd4b41038e0 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b2d9d700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b41038e0 0x7fd4b419f940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac004cb0 tx=0x7fd4ac004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac01f030 con 0x7fd4b41038e0 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b4101470 con 0x7fd4b41038e0 2024-04-24T23:54:10.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4ac005040 con 0x7fd4b41038e0 2024-04-24T23:54:10.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b4101950 con 0x7fd4b41038e0 2024-04-24T23:54:10.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.628+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac01b690 con 0x7fd4b41038e0 2024-04-24T23:54:10.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.629+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd4ac022030 con 0x7fd4b41038e0 2024-04-24T23:54:10.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.629+0000 7fd4a37fe700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd49c039330 0x7fd49c03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:10.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.629+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd4ac04de50 con 0x7fd4b41038e0 2024-04-24T23:54:10.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.629+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd494005360 con 0x7fd4b41038e0 2024-04-24T23:54:10.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.629+0000 7fd4b259c700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd49c039330 0x7fd49c03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:10.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.630+0000 7fd4b259c700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd49c039330 0x7fd49c03b7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd4b406a680 tx=0x7fd4a4013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:54:10.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.634+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd4ac018d70 con 0x7fd4b41038e0 2024-04-24T23:54:10.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:10.840+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd494000bf0 con 0x7fd49c039330 2024-04-24T23:54:11.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:10.783+0000 7fc66f125200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:54:11.226 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:10.892+0000 7ff59c831200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:54:11.227 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:10.986+0000 7ff59c831200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:54:11.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:11.416+0000 7fc66f125200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:54:11.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:11.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:11.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:11.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:11.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:11.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:11.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:11.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:11.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:11.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:11.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:11.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:12.159 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:11.859+0000 7fc66f125200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:54:12.159 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:11.958+0000 7fc66f125200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:54:12.159 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:12.159+0000 7fc66f125200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:54:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:12.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:12.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:12.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:12.548 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:12.210+0000 7ff59c831200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:54:12.548 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:12.378+0000 7ff59c831200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:54:12.548 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:12.548+0000 7ff59c831200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:54:12.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:12.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.548 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:12.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:12.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:12.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:54:12.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:54:12.549 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:13.009 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:12.680+0000 7fc66f125200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:54:13.009 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:12.770+0000 7fc66f125200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:54:13.101 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:12.832+0000 7ff59c831200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:54:13.101 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:12.923+0000 7ff59c831200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:54:13.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:13.009+0000 7fc66f125200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:54:13.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:13.142+0000 7fc66f125200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:54:13.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:13.236+0000 7fc66f125200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:54:13.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:13 smithi173 ceph-mon[32402]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:54:13.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:13 smithi173 ceph-mon[32402]: Reconfiguring daemon mgr.x on smithi110 2024-04-24T23:54:13.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:13 smithi173 ceph-mon[32402]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:13.602 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:13.127+0000 7ff59c831200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:54:13.602 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:13.316+0000 7ff59c831200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:54:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:13 smithi188 ceph-mon[33285]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:54:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:13 smithi188 ceph-mon[33285]: Reconfiguring daemon mgr.x on smithi110 2024-04-24T23:54:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:13 smithi188 ceph-mon[33285]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:13.767 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:13.352+0000 7fc66f125200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:54:13.767 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:13.440+0000 7fc66f125200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:54:13.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:13 smithi110 ceph-mon[29187]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:54:13.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:13 smithi110 ceph-mon[29187]: Reconfiguring daemon mgr.x on smithi110 2024-04-24T23:54:13.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:13 smithi110 ceph-mon[29187]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:14.288 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:13.905+0000 7ff59c831200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:54:14.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:14.243+0000 7fc66f125200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:54:14.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 24 23:54:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z[35147]: 2024-04-24T23:54:14.454+0000 7fc66f125200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:54:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:14 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/51175098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6eba556-dc44-4043-a0d7-c9d6dc839473"}]: dispatch 2024-04-24T23:54:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:14 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/51175098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6eba556-dc44-4043-a0d7-c9d6dc839473"}]': finished 2024-04-24T23:54:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:14 smithi188 ceph-mon[33285]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:54:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:14 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3918370506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:54:14.601 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:14.323+0000 7ff59c831200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:54:14.601 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:14.415+0000 7ff59c831200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:54:14.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:14 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/51175098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6eba556-dc44-4043-a0d7-c9d6dc839473"}]: dispatch 2024-04-24T23:54:14.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:14 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/51175098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6eba556-dc44-4043-a0d7-c9d6dc839473"}]': finished 2024-04-24T23:54:14.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:14 smithi173 ceph-mon[32402]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:54:14.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:14.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:14 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3918370506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:54:14.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:14 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/51175098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6eba556-dc44-4043-a0d7-c9d6dc839473"}]: dispatch 2024-04-24T23:54:14.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:14 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/51175098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6eba556-dc44-4043-a0d7-c9d6dc839473"}]': finished 2024-04-24T23:54:14.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:14 smithi110 ceph-mon[29187]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:54:14.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:14.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:14 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3918370506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:54:15.101 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:14.613+0000 7ff59c831200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:54:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:15.297+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7fd4ac01b830 con 0x7fd4b41038e0 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:15.133+0000 7ff59c831200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:15.222+0000 7ff59c831200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-mon[32402]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-mon[32402]: Standby manager daemon z started 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-mon[32402]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-mon[32402]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-mon[32402]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-mon[32402]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:54:15.460 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:15 smithi188 ceph-mon[33285]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:15 smithi188 ceph-mon[33285]: Standby manager daemon z started 2024-04-24T23:54:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:15 smithi188 ceph-mon[33285]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:54:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:15 smithi188 ceph-mon[33285]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:54:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:15 smithi188 ceph-mon[33285]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:54:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:15 smithi188 ceph-mon[33285]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:54:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:15 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:15.779 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:15.460+0000 7ff59c831200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:54:15.779 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:15.591+0000 7ff59c831200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:54:15.779 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:15.683+0000 7ff59c831200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:54:15.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:15 smithi110 ceph-mon[29187]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:15.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:15 smithi110 ceph-mon[29187]: Standby manager daemon z started 2024-04-24T23:54:15.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:15 smithi110 ceph-mon[29187]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:54:15.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:15 smithi110 ceph-mon[29187]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:54:15.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:15 smithi110 ceph-mon[29187]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:54:15.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:15 smithi110 ceph-mon[29187]: from='mgr.? 172.21.15.188:0/2069298063' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:54:15.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:15 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:16.101 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:15.799+0000 7ff59c831200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:54:16.101 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:15.890+0000 7ff59c831200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:54:16.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:16 smithi173 ceph-mon[32402]: mgrmap e14: x(active, since 80s), standbys: z 2024-04-24T23:54:16.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:16 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:54:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:16 smithi188 ceph-mon[33285]: mgrmap e14: x(active, since 80s), standbys: z 2024-04-24T23:54:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:16 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:54:16.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:16 smithi110 ceph-mon[29187]: mgrmap e14: x(active, since 80s), standbys: z 2024-04-24T23:54:16.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:16 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:54:17.101 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:16.704+0000 7ff59c831200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:54:17.101 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 24 23:54:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y[33792]: 2024-04-24T23:54:16.916+0000 7ff59c831200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:54:17.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:17.306+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fd4ac0182b0 con 0x7fd4b41038e0 2024-04-24T23:54:17.530 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:17 smithi110 ceph-mon[29187]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:17.530 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:17 smithi110 ceph-mon[29187]: Standby manager daemon y started 2024-04-24T23:54:17.530 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:17 smithi110 ceph-mon[29187]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:54:17.530 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:17 smithi110 ceph-mon[29187]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:54:17.530 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:17 smithi110 ceph-mon[29187]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:54:17.530 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:17 smithi110 ceph-mon[29187]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:54:17.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:17 smithi173 ceph-mon[32402]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:17.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:17 smithi173 ceph-mon[32402]: Standby manager daemon y started 2024-04-24T23:54:17.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:17 smithi173 ceph-mon[32402]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:54:17.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:17 smithi173 ceph-mon[32402]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:54:17.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:17 smithi173 ceph-mon[32402]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:54:17.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:17 smithi173 ceph-mon[32402]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:54:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:17 smithi188 ceph-mon[33285]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:17 smithi188 ceph-mon[33285]: Standby manager daemon y started 2024-04-24T23:54:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:17 smithi188 ceph-mon[33285]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:54:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:17 smithi188 ceph-mon[33285]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:54:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:17 smithi188 ceph-mon[33285]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:54:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:17 smithi188 ceph-mon[33285]: from='mgr.? 172.21.15.173:0/594351724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:54:18.400 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:18 smithi110 ceph-mon[29187]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-24T23:54:18.400 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:18 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:54:18.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:18 smithi173 ceph-mon[32402]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-24T23:54:18.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:18 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:54:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:18 smithi188 ceph-mon[33285]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-24T23:54:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:18 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:54:19.373 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:19 smithi110 ceph-mon[29187]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:19.373 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:19.373 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:19.373 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:19.373 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:19.373 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:19.373 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:19.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:19 smithi173 ceph-mon[32402]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:19.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:19.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:19.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:19.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:19.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:19.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:19 smithi188 ceph-mon[33285]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:21 smithi188 ceph-mon[33285]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:21.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:21 smithi110 ceph-mon[29187]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:21.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:21 smithi173 ceph-mon[32402]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:54:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:22 smithi188 ceph-mon[33285]: Deploying daemon osd.0 on smithi110 2024-04-24T23:54:22.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:54:22.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:22 smithi110 ceph-mon[29187]: Deploying daemon osd.0 on smithi110 2024-04-24T23:54:22.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:54:22.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:22.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:22 smithi173 ceph-mon[32402]: Deploying daemon osd.0 on smithi110 2024-04-24T23:54:23.753 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:23 smithi110 ceph-mon[29187]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:23.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:23 smithi188 ceph-mon[33285]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:23.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:23 smithi173 ceph-mon[32402]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:25 smithi188 ceph-mon[33285]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:25.769 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:25 smithi110 ceph-mon[29187]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:25.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:25 smithi173 ceph-mon[32402]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:27.601 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:27 smithi110 ceph-mon[29187]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:27.601 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:27.601 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:27.601 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:27 smithi188 ceph-mon[33285]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:27.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:27 smithi173 ceph-mon[32402]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:27.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:27.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:27.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:29.647 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:29 smithi110 ceph-mon[29187]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:29 smithi188 ceph-mon[33285]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:29.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:29 smithi173 ceph-mon[32402]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:29.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.952+0000 7fd4a37fe700 1 -- 172.21.15.110:0/3869796963 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd494000bf0 con 0x7fd49c039330 2024-04-24T23:54:29.953 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 0 on host 'smithi110' 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd49c039330 msgr2=0x7fd49c03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd49c039330 0x7fd49c03b7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd4b406a680 tx=0x7fd4a4013040 comp rx=0 tx=0).stop 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b41038e0 msgr2=0x7fd4b419f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b41038e0 0x7fd4b419f940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac004cb0 tx=0x7fd4ac004ce0 comp rx=0 tx=0).stop 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 shutdown_connections 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd49c039330 0x7fd49c03b7f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b41038e0 0x7fd4b419f940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4b4100360 0x7fd4b4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 --2- 172.21.15.110:0/3869796963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4b4100d20 0x7fd4b41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 >> 172.21.15.110:0/3869796963 conn(0x7fd4b40fbf00 msgr2=0x7fd4b4106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 shutdown_connections 2024-04-24T23:54:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:54:29.955+0000 7fd4b8965700 1 -- 172.21.15.110:0/3869796963 wait complete. 2024-04-24T23:54:30.940 DEBUG:teuthology.orchestra.run.smithi110:osd.0> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T23:54:30.942 INFO:tasks.cephadm:Deploying osd.1 on smithi173 with /dev/vg_nvme/lv_4... 2024-04-24T23:54:30.942 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:54:31.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:30 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:30 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:30 smithi110 ceph-mon[29187]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:31.164 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:30 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.164 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:30 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.164 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:30 smithi110 ceph-mon[29187]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:54:31.195 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:54:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:30 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:30 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:30 smithi188 ceph-mon[33285]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:30 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:30 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:30 smithi188 ceph-mon[33285]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:54:31.316 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:30 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.317 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:30 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.317 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:30 smithi173 ceph-mon[32402]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:31.317 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:30 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.317 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:30 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:31.317 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:30 smithi173 ceph-mon[32402]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:54:32.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:32 smithi110 ceph-mon[29187]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:54:32.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:32 smithi110 ceph-mon[29187]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:54:32.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:32 smithi110 ceph-mon[29187]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:54:32.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:32.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:54:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0[44124]: 2024-04-24T23:54:32.557+0000 7f8fc257b700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:54:32.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:54:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0[44124]: 2024-04-24T23:54:32.560+0000 7f8fbe36e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:54:32.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:32 smithi173 ceph-mon[32402]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:54:32.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:32 smithi173 ceph-mon[32402]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:54:32.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:32 smithi173 ceph-mon[32402]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:54:32.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:32.913 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:54:32.913 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:54:32.913 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-24T23:54:32.913 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-24T23:54:32.913 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0158121 s, 663 MB/s 2024-04-24T23:54:32.913 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-24T23:54:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:32 smithi188 ceph-mon[33285]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:54:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:32 smithi188 ceph-mon[33285]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:54:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:32 smithi188 ceph-mon[33285]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:54:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:33.500 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2024-04-24T23:54:33.769 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:54:33.820 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:33 smithi173 ceph-mon[32402]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:33.820 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:33 smithi173 ceph-mon[32402]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:54:33.821 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:33 smithi173 ceph-mon[32402]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:54:33.821 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:33.821 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:33 smithi110 ceph-mon[29187]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:33 smithi110 ceph-mon[29187]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:54:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:33 smithi110 ceph-mon[29187]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:54:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:33.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:33 smithi188 ceph-mon[33285]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:33 smithi188 ceph-mon[33285]: from='osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:54:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:33 smithi188 ceph-mon[33285]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:54:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] boot 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:34.723 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:34.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:54:34.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:54:34.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] boot 2024-04-24T23:54:34.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:54:34.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:34.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:34.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:34.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:54:34.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:34.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:34.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:34.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:34.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: osd.0 [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] boot 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:35.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 -- 172.21.15.173:0/232796273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 msgr2=0x7f9358101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 --2- 172.21.15.173:0/232796273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 0x7f9358101910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f934c009990 tx=0x7f934c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 -- 172.21.15.173:0/232796273 shutdown_connections 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 --2- 172.21.15.173:0/232796273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9358101e50 0x7f93581096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 --2- 172.21.15.173:0/232796273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93581027e0 0x7f9358102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 --2- 172.21.15.173:0/232796273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 0x7f9358101910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 -- 172.21.15.173:0/232796273 >> 172.21.15.173:0/232796273 conn(0x7f93580fbf00 msgr2=0x7f93580fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 -- 172.21.15.173:0/232796273 shutdown_connections 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 -- 172.21.15.173:0/232796273 wait complete. 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.221+0000 7f935d12f700 1 Processor -- start 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 -- start start 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 0x7f93581051e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:35.223 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9358101e50 0x7f9358103880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93581027e0 0x7f9358103dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f935810cd20 con 0x7f93581027e0 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f935810cba0 con 0x7f9358101490 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f935810cea0 con 0x7f9358101e50 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 0x7f93581051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 0x7f93581051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:41088/0 (socket says 172.21.15.173:41088) 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 -- 172.21.15.173:0/2813020392 learned_addr learned my addr 172.21.15.173:0/2813020392 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935659c700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9358101e50 0x7f9358103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935759e700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93581027e0 0x7f9358103dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 -- 172.21.15.173:0/2813020392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9358101e50 msgr2=0x7f9358103880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9358101e50 0x7f9358103880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 -- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93581027e0 msgr2=0x7f9358103dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93581027e0 0x7f9358103dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:35.224 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 -- 172.21.15.173:0/2813020392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9358104690 con 0x7f9358101490 2024-04-24T23:54:35.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f9356d9d700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 0x7f93581051e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f934800c8f0 tx=0x7f934800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:54:35.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f933ffff700 1 -- 172.21.15.173:0/2813020392 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348004220 con 0x7f9358101490 2024-04-24T23:54:35.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f933ffff700 1 -- 172.21.15.173:0/2813020392 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9348007db0 con 0x7f9358101490 2024-04-24T23:54:35.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93581a25f0 con 0x7f9358101490 2024-04-24T23:54:35.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f933ffff700 1 -- 172.21.15.173:0/2813020392 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348003d10 con 0x7f9358101490 2024-04-24T23:54:35.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.222+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93581a2ab0 con 0x7f9358101490 2024-04-24T23:54:35.226 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.223+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935810d4e0 con 0x7f9358101490 2024-04-24T23:54:35.226 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.224+0000 7f933ffff700 1 -- 172.21.15.173:0/2813020392 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f9348007900 con 0x7f9358101490 2024-04-24T23:54:35.226 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.225+0000 7f933ffff700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93400a2b90 0x7f93400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:35.226 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.225+0000 7f933ffff700 1 -- 172.21.15.173:0/2813020392 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f93480cd2f0 con 0x7f9358101490 2024-04-24T23:54:35.226 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.225+0000 7f935659c700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93400a2b90 0x7f93400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:35.226 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.226+0000 7f935659c700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93400a2b90 0x7f93400a5050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f934c0038b0 tx=0x7f934c003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:54:35.229 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.230+0000 7f933ffff700 1 -- 172.21.15.173:0/2813020392 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9348014030 con 0x7f9358101490 2024-04-24T23:54:35.432 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:35.432+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f93580fe0f0 con 0x7f93400a2b90 2024-04-24T23:54:35.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi110 2024-04-24T23:54:35.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi110 to 17088M 2024-04-24T23:54:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:54:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi110 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi110 to 17088M 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi110 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi110 to 17088M 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:36.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:36 smithi173 ceph-mon[32402]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:36 smithi188 ceph-mon[33285]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:37.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:36 smithi110 ceph-mon[29187]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:54:37.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:37 smithi173 ceph-mon[32402]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:37 smithi188 ceph-mon[33285]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:38.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:37 smithi110 ceph-mon[29187]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:38.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:38 smithi173 ceph-mon[32402]: from='client.? 172.21.15.173:0/3119350681' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]: dispatch 2024-04-24T23:54:38.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:38 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]: dispatch 2024-04-24T23:54:38.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:38 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]': finished 2024-04-24T23:54:38.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:38 smithi173 ceph-mon[32402]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:54:38.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:38 smithi188 ceph-mon[33285]: from='client.? 172.21.15.173:0/3119350681' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]: dispatch 2024-04-24T23:54:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:38 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]: dispatch 2024-04-24T23:54:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:38 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]': finished 2024-04-24T23:54:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:38 smithi188 ceph-mon[33285]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:54:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:39.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:38 smithi110 ceph-mon[29187]: from='client.? 172.21.15.173:0/3119350681' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]: dispatch 2024-04-24T23:54:39.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:38 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]: dispatch 2024-04-24T23:54:39.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:38 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbfc4554-e9a2-4cc3-a392-cfe42b675b63"}]': finished 2024-04-24T23:54:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:38 smithi110 ceph-mon[29187]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:54:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:39.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:39 smithi173 ceph-mon[32402]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:39.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:39 smithi173 ceph-mon[32402]: from='client.? 172.21.15.173:0/1932847731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:54:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:39 smithi188 ceph-mon[33285]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:39 smithi188 ceph-mon[33285]: from='client.? 172.21.15.173:0/1932847731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:54:40.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:39 smithi110 ceph-mon[29187]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:40.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:39 smithi110 ceph-mon[29187]: from='client.? 172.21.15.173:0/1932847731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:54:41.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:41 smithi173 ceph-mon[32402]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:41 smithi188 ceph-mon[33285]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:42.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:41 smithi110 ceph-mon[29187]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:43.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:43 smithi173 ceph-mon[32402]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:43 smithi188 ceph-mon[33285]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:44.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:43 smithi110 ceph-mon[29187]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:45.813 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:45 smithi173 ceph-mon[32402]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:45.813 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:45 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:54:45.813 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:45 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:45 smithi188 ceph-mon[33285]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:45 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:54:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:45 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:46.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:45 smithi110 ceph-mon[29187]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:46.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:45 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:54:46.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:45 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:46.775 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:46 smithi173 ceph-mon[32402]: Deploying daemon osd.1 on smithi173 2024-04-24T23:54:47.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:46 smithi188 ceph-mon[33285]: Deploying daemon osd.1 on smithi173 2024-04-24T23:54:47.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:46 smithi110 ceph-mon[29187]: Deploying daemon osd.1 on smithi173 2024-04-24T23:54:47.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:47 smithi173 ceph-mon[32402]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:47.937 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:47 smithi110 ceph-mon[29187]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:47 smithi188 ceph-mon[33285]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:49.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:49 smithi173 ceph-mon[32402]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:49 smithi188 ceph-mon[33285]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:49 smithi110 ceph-mon[29187]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:51.447 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:51.447 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:51.448 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:51.448 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:51 smithi173 ceph-mon[32402]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:51.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:51.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:51.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:51.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:51 smithi188 ceph-mon[33285]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:51.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:51.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:51.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:51.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:51 smithi110 ceph-mon[29187]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:53.495 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 1 on host 'smithi173' 2024-04-24T23:54:53.495 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.495+0000 7f933ffff700 1 -- 172.21.15.173:0/2813020392 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f93580fe0f0 con 0x7f93400a2b90 2024-04-24T23:54:53.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93400a2b90 msgr2=0x7f93400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93400a2b90 0x7f93400a5050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f934c0038b0 tx=0x7f934c003680 comp rx=0 tx=0).stop 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 msgr2=0x7f93581051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 0x7f93581051e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f934800c8f0 tx=0x7f934800ccb0 comp rx=0 tx=0).stop 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 shutdown_connections 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f93400a2b90 0x7f93400a5050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93581027e0 0x7f9358103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9358101490 0x7f93581051e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 --2- 172.21.15.173:0/2813020392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9358101e50 0x7f9358103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 >> 172.21.15.173:0/2813020392 conn(0x7f93580fbf00 msgr2=0x7f93580fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 shutdown_connections 2024-04-24T23:54:53.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:54:53.497+0000 7f935d12f700 1 -- 172.21.15.173:0/2813020392 wait complete. 2024-04-24T23:54:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:53 smithi188 ceph-mon[33285]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:53 smithi110 ceph-mon[29187]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:53 smithi173 ceph-mon[32402]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:53.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.819 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.819 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:53.819 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:53.819 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.819 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:53.819 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:54.212 DEBUG:teuthology.orchestra.run.smithi173:osd.1> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T23:54:54.214 INFO:tasks.cephadm:Deploying osd.2 on smithi188 with /dev/vg_nvme/lv_4... 2024-04-24T23:54:54.215 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:54:54.457 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:54:54.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:54.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:54 smithi188 ceph-mon[33285]: from='osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:54:54.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:54 smithi188 ceph-mon[33285]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:54:54.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:54 smithi110 ceph-mon[29187]: from='osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:54:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:54 smithi110 ceph-mon[29187]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:54:54.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:54:54.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:54 smithi173 ceph-mon[32402]: from='osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:54:54.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:54 smithi173 ceph-mon[32402]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:54:55.687 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:55 smithi188 ceph-mon[33285]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:55.687 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:55 smithi188 ceph-mon[33285]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:54:55.687 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:55 smithi188 ceph-mon[33285]: from='osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:54:55.687 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:55 smithi188 ceph-mon[33285]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:54:55.687 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:55.687 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:55 smithi188 ceph-mon[33285]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:54:55.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:55 smithi173 ceph-mon[32402]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:55.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:55 smithi173 ceph-mon[32402]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:54:55.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:55 smithi173 ceph-mon[32402]: from='osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:54:55.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:55 smithi173 ceph-mon[32402]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:54:55.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:55.708 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:55 smithi173 ceph-mon[32402]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:54:55.708 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:54:55 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1[37000]: 2024-04-24T23:54:55.518+0000 7f76a84a2700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:54:55.709 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:54:55 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1[37000]: 2024-04-24T23:54:55.521+0000 7f76a4295700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:54:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:55 smithi110 ceph-mon[29187]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:55 smithi110 ceph-mon[29187]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:54:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:55 smithi110 ceph-mon[29187]: from='osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:54:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:55 smithi110 ceph-mon[29187]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:54:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:55 smithi110 ceph-mon[29187]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:54:56.297 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:54:56.297 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:54:56.297 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-04-24T23:54:56.297 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-04-24T23:54:56.297 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133902 s, 783 MB/s 2024-04-24T23:54:56.297 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-04-24T23:54:56.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:56 smithi188 ceph-mon[33285]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:54:56.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:56 smithi188 ceph-mon[33285]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:54:56.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:56.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:56.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:56 smithi110 ceph-mon[29187]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:54:56.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:56 smithi110 ceph-mon[29187]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:54:56.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:56.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:56.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:56 smithi173 ceph-mon[32402]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:54:56.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:56 smithi173 ceph-mon[32402]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:54:56.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:56.840 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:56.932 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_4 2024-04-24T23:54:57.169 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:54:57.520 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:54:57.520 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:54:57.520 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:54:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:54:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] boot 2024-04-24T23:54:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:54:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:54:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] boot 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:57.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:57.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: osd.1 [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] boot 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:54:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:54:58.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.586+0000 7f4d8576b700 1 -- 172.21.15.188:0/3626476565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 msgr2=0x7f4d80105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:58.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.586+0000 7f4d8576b700 1 --2- 172.21.15.188:0/3626476565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 0x7f4d80105100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009990 tx=0x7f4d70009ca0 comp rx=0 tx=0).stop 2024-04-24T23:54:58.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.586+0000 7f4d8576b700 1 -- 172.21.15.188:0/3626476565 shutdown_connections 2024-04-24T23:54:58.588 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.586+0000 7f4d8576b700 1 --2- 172.21.15.188:0/3626476565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d80107f70 0x7f4d8010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.586+0000 7f4d8576b700 1 --2- 172.21.15.188:0/3626476565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 0x7f4d80105100 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.586+0000 7f4d8576b700 1 --2- 172.21.15.188:0/3626476565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d80105640 0x7f4d80107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.586+0000 7f4d8576b700 1 -- 172.21.15.188:0/3626476565 >> 172.21.15.188:0/3626476565 conn(0x7f4d800fc720 msgr2=0x7f4d800feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 -- 172.21.15.188:0/3626476565 shutdown_connections 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 -- 172.21.15.188:0/3626476565 wait complete. 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 Processor -- start 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 -- start start 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 0x7f4d80102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d80105640 0x7f4d80100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d80107f70 0x7f4d80101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010dd10 con 0x7f4d80105640 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010db90 con 0x7f4d80107f70 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d8576b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010de90 con 0x7f4d80102d10 2024-04-24T23:54:58.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.587+0000 7f4d7effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 0x7f4d80102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 0x7f4d80102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:47814/0 (socket says 172.21.15.188:47814) 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7effd700 1 -- 172.21.15.188:0/86597512 learned_addr learned my addr 172.21.15.188:0/86597512 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7f7fe700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d80107f70 0x7f4d80101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7e7fc700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d80105640 0x7f4d80100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7effd700 1 -- 172.21.15.188:0/86597512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d80107f70 msgr2=0x7f4d80101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7effd700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d80107f70 0x7f4d80101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7effd700 1 -- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d80105640 msgr2=0x7f4d80100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7effd700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d80105640 0x7f4d80100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7effd700 1 -- 172.21.15.188:0/86597512 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d80101ab0 con 0x7f4d80102d10 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d7effd700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 0x7f4d80102670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000b5c0 tx=0x7f4d700047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d67fff700 1 -- 172.21.15.188:0/86597512 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70003ca0 con 0x7f4d80102d10 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d801a2510 con 0x7f4d80102d10 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d67fff700 1 -- 172.21.15.188:0/86597512 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d70004830 con 0x7f4d80102d10 2024-04-24T23:54:58.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d67fff700 1 -- 172.21.15.188:0/86597512 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70005480 con 0x7f4d80102d10 2024-04-24T23:54:58.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.588+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801a2900 con 0x7f4d80102d10 2024-04-24T23:54:58.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.589+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d80068f20 con 0x7f4d80102d10 2024-04-24T23:54:58.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.590+0000 7f4d67fff700 1 -- 172.21.15.188:0/86597512 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f4d70029060 con 0x7f4d80102d10 2024-04-24T23:54:58.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.591+0000 7f4d67fff700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d680a2b90 0x7f4d680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:54:58.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.591+0000 7f4d67fff700 1 -- 172.21.15.188:0/86597512 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f4d700ce960 con 0x7f4d80102d10 2024-04-24T23:54:58.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.591+0000 7f4d7e7fc700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d680a2b90 0x7f4d680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:54:58.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.592+0000 7f4d7e7fc700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d680a2b90 0x7f4d680a5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d74005950 tx=0x7f4d740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:54:58.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.595+0000 7f4d67fff700 1 -- 172.21.15.188:0/86597512 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d70013070 con 0x7f4d80102d10 2024-04-24T23:54:58.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:58 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi173 2024-04-24T23:54:58.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:58 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-24T23:54:58.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:58 smithi188 ceph-mon[33285]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:54:58.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:58 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi173 2024-04-24T23:54:58.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:58 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-24T23:54:58.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:58 smithi110 ceph-mon[29187]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:54:58.798 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:54:58.798+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4d80062670 con 0x7f4d680a2b90 2024-04-24T23:54:58.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:58 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi173 2024-04-24T23:54:58.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:58 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-24T23:54:58.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:58 smithi173 ceph-mon[32402]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:54:59.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:59 smithi188 ceph-mon[33285]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:54:59.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:59.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:59.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:54:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:59.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:59 smithi110 ceph-mon[29187]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:54:59.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:59.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:59.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:54:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:54:59.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:59 smithi173 ceph-mon[32402]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:54:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:54:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:54:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:54:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:00.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:00 smithi188 ceph-mon[33285]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:00.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:00 smithi110 ceph-mon[29187]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:00.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:00 smithi173 ceph-mon[32402]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:01.802 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:01 smithi188 ceph-mon[33285]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:01.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:01 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/2698015347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]: dispatch 2024-04-24T23:55:01.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:01 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]: dispatch 2024-04-24T23:55:01.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:01 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]': finished 2024-04-24T23:55:01.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:01 smithi188 ceph-mon[33285]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:55:01.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:01 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:01.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:01 smithi173 ceph-mon[32402]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:01.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:01 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/2698015347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]: dispatch 2024-04-24T23:55:01.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:01 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]: dispatch 2024-04-24T23:55:01.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:01 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]': finished 2024-04-24T23:55:01.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:01 smithi173 ceph-mon[32402]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:55:01.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:01 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:01 smithi110 ceph-mon[29187]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:01 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/2698015347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]: dispatch 2024-04-24T23:55:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:01 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]: dispatch 2024-04-24T23:55:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:01 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c3e4437-7088-4060-a771-70bebf386247"}]': finished 2024-04-24T23:55:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:01 smithi110 ceph-mon[29187]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:55:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:01 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:02.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:02 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/3155807387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:02 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/3155807387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:03.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:02 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/3155807387' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:03.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:03 smithi173 ceph-mon[32402]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:03 smithi188 ceph-mon[33285]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:04.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:03 smithi110 ceph-mon[29187]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:05.697 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:05 smithi188 ceph-mon[33285]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:05.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:05 smithi173 ceph-mon[32402]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:06.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:05 smithi110 ceph-mon[29187]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:07.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:07 smithi173 ceph-mon[32402]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:08.008 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:07 smithi188 ceph-mon[33285]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:08.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:07 smithi110 ceph-mon[29187]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:09.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:09 smithi173 ceph-mon[32402]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:09 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:55:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:09 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:09 smithi173 ceph-mon[32402]: Deploying daemon osd.2 on smithi188 2024-04-24T23:55:09.907 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:09 smithi188 ceph-mon[33285]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:09.907 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:09 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:55:09.907 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:09 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:09.907 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:09 smithi188 ceph-mon[33285]: Deploying daemon osd.2 on smithi188 2024-04-24T23:55:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:09 smithi110 ceph-mon[29187]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:55:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:09 smithi110 ceph-mon[29187]: Deploying daemon osd.2 on smithi188 2024-04-24T23:55:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:11 smithi188 ceph-mon[33285]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:11.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:11 smithi173 ceph-mon[32402]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:12.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:11 smithi110 ceph-mon[29187]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:13.673 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:13 smithi188 ceph-mon[33285]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:13.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:13 smithi173 ceph-mon[32402]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:14.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:13 smithi110 ceph-mon[29187]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:15.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:15 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:15.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:15 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:15.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:15 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:15.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:15 smithi188 ceph-mon[33285]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:15.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:15.285 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:15 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:15.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:15 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:15.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:15 smithi110 ceph-mon[29187]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:15.351 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:15 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:15.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:15 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:15.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:15 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:15.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:15 smithi173 ceph-mon[32402]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:17.625 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:17 smithi188 ceph-mon[33285]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:17.625 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:17 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:17.625 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:17 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:17.625 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:17 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:17.625 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:17 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:17.625 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:17 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:17.665 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.665+0000 7f4d67fff700 1 -- 172.21.15.188:0/86597512 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4d80062670 con 0x7f4d680a2b90 2024-04-24T23:55:17.665 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 2 on host 'smithi188' 2024-04-24T23:55:17.667 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d680a2b90 msgr2=0x7f4d680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:17.667 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d680a2b90 0x7f4d680a5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d74005950 tx=0x7f4d740058e0 comp rx=0 tx=0).stop 2024-04-24T23:55:17.667 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 msgr2=0x7f4d80102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:17.667 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 0x7f4d80102670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000b5c0 tx=0x7f4d700047b0 comp rx=0 tx=0).stop 2024-04-24T23:55:17.667 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 shutdown_connections 2024-04-24T23:55:17.668 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d680a2b90 0x7f4d680a5050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:17.668 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d80107f70 0x7f4d80101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:17.668 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d80102d10 0x7f4d80102670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:17.668 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 --2- 172.21.15.188:0/86597512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d80105640 0x7f4d80100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:17.668 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.667+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 >> 172.21.15.188:0/86597512 conn(0x7f4d800fc720 msgr2=0x7f4d801037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:55:17.668 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.668+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 shutdown_connections 2024-04-24T23:55:17.668 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:55:17.668+0000 7f4d8576b700 1 -- 172.21.15.188:0/86597512 wait complete. 2024-04-24T23:55:17.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:17 smithi173 ceph-mon[32402]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:17 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:17 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:17 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:17 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:17 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:17.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:17 smithi110 ceph-mon[29187]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:17.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:17 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:17.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:17 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:17.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:17 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:17.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:17 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:17.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:17 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:18.230 DEBUG:teuthology.orchestra.run.smithi188:osd.2> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T23:55:18.232 INFO:tasks.cephadm:Deploying osd.3 on smithi110 with /dev/vg_nvme/lv_3... 2024-04-24T23:55:18.232 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:55:18.476 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:55:18.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:18 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:18.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:18 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:18.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:18 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:18.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:18 smithi110 ceph-mon[29187]: from='osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:55:18.869 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:18 smithi110 ceph-mon[29187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:55:18.870 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:18 smithi110 ceph-mon[29187]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:18 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:18 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:18 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:18 smithi188 ceph-mon[33285]: from='osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:55:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:18 smithi188 ceph-mon[33285]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:55:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:18 smithi188 ceph-mon[33285]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:19.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:18 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:18 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:18 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:18 smithi173 ceph-mon[32402]: from='osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:55:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:18 smithi173 ceph-mon[32402]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:55:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:18 smithi173 ceph-mon[32402]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:19.726 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:19 smithi188 ceph-mon[33285]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:55:19.726 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:19 smithi188 ceph-mon[33285]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:55:19.726 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:19.726 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:19 smithi188 ceph-mon[33285]: from='osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:55:19.726 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:19 smithi188 ceph-mon[33285]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:55:19.947 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:19 smithi110 ceph-mon[29187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:55:19.947 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:19 smithi110 ceph-mon[29187]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:55:19.947 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:19.947 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:19 smithi110 ceph-mon[29187]: from='osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:55:19.947 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:19 smithi110 ceph-mon[29187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:55:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:19 smithi173 ceph-mon[32402]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:55:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:19 smithi173 ceph-mon[32402]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:55:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:19 smithi173 ceph-mon[32402]: from='osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:55:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:19 smithi173 ceph-mon[32402]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:55:20.515 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:55:20.515 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:55:20.516 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-04-24T23:55:20.516 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-04-24T23:55:20.516 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00854045 s, 1.2 GB/s 2024-04-24T23:55:20.516 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-04-24T23:55:20.517 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2[38443]: 2024-04-24T23:55:20.079+0000 7fcc103c8700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:55:20.517 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2[38443]: 2024-04-24T23:55:20.083+0000 7fcc081b3700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:55:20.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:20 smithi110 ceph-mon[29187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:55:20.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:20 smithi110 ceph-mon[29187]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:55:20.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:20 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:20.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:20 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:20.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:20 smithi110 ceph-mon[29187]: from='osd.2 ' entity='osd.2' 2024-04-24T23:55:20.881 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:20 smithi110 ceph-mon[29187]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:20.917 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:20 smithi188 ceph-mon[33285]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:55:20.917 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:20 smithi188 ceph-mon[33285]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:55:20.917 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:20 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:20.917 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:20 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:20.917 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:20 smithi188 ceph-mon[33285]: from='osd.2 ' entity='osd.2' 2024-04-24T23:55:20.917 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:20 smithi188 ceph-mon[33285]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:21.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:20 smithi173 ceph-mon[32402]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:55:21.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:20 smithi173 ceph-mon[32402]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:55:21.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:20 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:21.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:20 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:21.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:20 smithi173 ceph-mon[32402]: from='osd.2 ' entity='osd.2' 2024-04-24T23:55:21.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:20 smithi173 ceph-mon[32402]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:55:21.185 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-04-24T23:55:21.431 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] boot 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi188 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:21.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:21.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi188 to 17088M 2024-04-24T23:55:21.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:21.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:21.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:21.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:55:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:55:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] boot 2024-04-24T23:55:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:55:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi188 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi188 to 17088M 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:22.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: osd.2 [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] boot 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi188 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi188 to 17088M 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:22.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:22.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:23.084 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:22 smithi110 ceph-mon[29187]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:23.084 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:55:23.084 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:55:23.084 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:22 smithi110 ceph-mon[29187]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:55:23.084 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:55:23.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.086+0000 7f4b8dea9700 1 -- 172.21.15.110:0/3839218292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b88100900 msgr2=0x7f4b88100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:23.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.086+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/3839218292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b88100900 0x7f4b88100d20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b780099c0 tx=0x7f4b78009cd0 comp rx=0 tx=0).stop 2024-04-24T23:55:23.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 -- 172.21.15.110:0/3839218292 shutdown_connections 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/3839218292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b881024a0 0x7f4b88108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/3839218292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b88100900 0x7f4b88100d20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/3839218292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b88101b00 0x7f4b88101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 -- 172.21.15.110:0/3839218292 >> 172.21.15.110:0/3839218292 conn(0x7f4b880fbec0 msgr2=0x7f4b880fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 -- 172.21.15.110:0/3839218292 shutdown_connections 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 -- 172.21.15.110:0/3839218292 wait complete. 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 Processor -- start 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 -- start start 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b88100900 0x7f4b88104840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b88101b00 0x7f4b88102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b881024a0 0x7f4b88103420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.087+0000 7f4b8dea9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810c470 con 0x7f4b881024a0 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b8dea9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810c2f0 con 0x7f4b88100900 2024-04-24T23:55:23.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b8dea9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810c5f0 con 0x7f4b88101b00 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b881024a0 0x7f4b88103420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b881024a0 0x7f4b88103420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48238/0 (socket says 172.21.15.110:48238) 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 -- 172.21.15.110:0/2608092920 learned_addr learned my addr 172.21.15.110:0/2608092920 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b877fe700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b88100900 0x7f4b88104840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b86ffd700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b88101b00 0x7f4b88102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 -- 172.21.15.110:0/2608092920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b88101b00 msgr2=0x7f4b88102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b88101b00 0x7f4b88102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 -- 172.21.15.110:0/2608092920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b88100900 msgr2=0x7f4b88104840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b88100900 0x7f4b88104840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 -- 172.21.15.110:0/2608092920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b88103cf0 con 0x7f4b881024a0 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b877fe700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b88100900 0x7f4b88104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b86ffd700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b88101b00 0x7f4b88102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:55:23.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b87fff700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b881024a0 0x7f4b88103420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000b970 tx=0x7f4b8000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:55:23.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b84ff9700 1 -- 172.21.15.110:0/2608092920 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80019070 con 0x7f4b881024a0 2024-04-24T23:55:23.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b84ff9700 1 -- 172.21.15.110:0/2608092920 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b80014440 con 0x7f4b881024a0 2024-04-24T23:55:23.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b881a26c0 con 0x7f4b881024a0 2024-04-24T23:55:23.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b84ff9700 1 -- 172.21.15.110:0/2608092920 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000ce60 con 0x7f4b881024a0 2024-04-24T23:55:23.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.088+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a2c50 con 0x7f4b881024a0 2024-04-24T23:55:23.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.089+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b88068f20 con 0x7f4b881024a0 2024-04-24T23:55:23.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.091+0000 7f4b84ff9700 1 -- 172.21.15.110:0/2608092920 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f4b8001c070 con 0x7f4b881024a0 2024-04-24T23:55:23.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.091+0000 7f4b84ff9700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b700a2b90 0x7f4b700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:55:23.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.092+0000 7f4b877fe700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b700a2b90 0x7f4b700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:55:23.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.092+0000 7f4b84ff9700 1 -- 172.21.15.110:0/2608092920 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f4b800cd470 con 0x7f4b881024a0 2024-04-24T23:55:23.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.092+0000 7f4b877fe700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b700a2b90 0x7f4b700a5050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b78000c00 tx=0x7f4b78005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:55:23.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.095+0000 7f4b84ff9700 1 -- 172.21.15.110:0/2608092920 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b8009b890 con 0x7f4b881024a0 2024-04-24T23:55:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:22 smithi173 ceph-mon[32402]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:55:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:55:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:22 smithi173 ceph-mon[32402]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:55:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:55:23.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:22 smithi188 ceph-mon[33285]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:23.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:55:23.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:55:23.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:22 smithi188 ceph-mon[33285]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:55:23.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:55:23.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:23.299+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4b88107f50 con 0x7f4b700a2b90 2024-04-24T23:55:24.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:23 smithi173 ceph-mon[32402]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:24.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:55:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:55:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:55:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:23 smithi173 ceph-mon[32402]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:55:24.103 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:23 smithi110 ceph-mon[29187]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:24.103 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:55:24.103 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:55:24.103 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:24.103 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:55:24.103 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:23 smithi110 ceph-mon[29187]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:55:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:23 smithi188 ceph-mon[33285]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:55:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:55:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:55:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:23 smithi188 ceph-mon[33285]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:55:24.361 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:55:24 smithi110 sudo[48490]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:55:24.361 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:55:24 smithi110 sudo[48490]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:24.361 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:55:24 smithi110 sudo[48490]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:24.361 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:55:24 smithi110 sudo[48490]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:24.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:55:24 smithi110 sudo[48506]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:55:24.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:55:24 smithi110 sudo[48506]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:24.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:55:24 smithi110 sudo[48506]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:24.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 24 23:55:24 smithi110 sudo[48506]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:24.883 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:55:24 smithi173 sudo[41107]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:55:24.883 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:55:24 smithi173 sudo[41107]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:24.883 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:55:24 smithi173 sudo[41107]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:24.883 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:55:24 smithi173 sudo[41107]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:25.351 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:55:24 smithi173 sudo[41110]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:55:25.352 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:55:24 smithi173 sudo[41110]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:25.352 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:55:24 smithi173 sudo[41110]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:25.352 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 24 23:55:24 smithi173 sudo[41110]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:25.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:25.401+0000 7f4b84ff9700 1 -- 172.21.15.110:0/2608092920 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4b80097800 con 0x7f4b881024a0 2024-04-24T23:55:25.412 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:25 smithi188 sudo[42338]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:55:25.412 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:25 smithi188 sudo[42338]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:25.412 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:25 smithi188 sudo[42338]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:25.413 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:25 smithi188 sudo[42338]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:25.413 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:25.413 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:25 smithi188 ceph-mon[33285]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:25.413 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:25 smithi188 ceph-mon[33285]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:55:25.413 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 ceph-mon[29187]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 ceph-mon[29187]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:55:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:25.767 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:25 smithi188 sudo[42341]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:55:25.767 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:25 smithi188 sudo[42341]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:25.767 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:25 smithi188 sudo[42341]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:25.767 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 24 23:55:25 smithi188 sudo[42341]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:25.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:25.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:25 smithi173 ceph-mon[32402]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:25.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:25 smithi173 ceph-mon[32402]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:55:25.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:25.948 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 sudo[48560]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:55:25.948 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 sudo[48560]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:25.948 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 sudo[48560]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:25.948 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 sudo[48560]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:26.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 sudo[48585]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:55:26.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 sudo[48585]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:26.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 sudo[48585]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:26.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:25 smithi110 sudo[48585]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:26.446 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 sudo[42344]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:55:26.446 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 sudo[42344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:26.446 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 sudo[42344]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:26.446 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 sudo[42344]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 sudo[42347]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:55:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 sudo[42347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 sudo[42347]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 sudo[42347]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:26 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1975915393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307060eb-b8bb-4b80-b43e-cc22c933400f"}]: dispatch 2024-04-24T23:55:26.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:55:26.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:26.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:26 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1975915393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307060eb-b8bb-4b80-b43e-cc22c933400f"}]: dispatch 2024-04-24T23:55:26.940 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 sudo[41113]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:55:26.940 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 sudo[41113]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:26.940 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 sudo[41113]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:26.941 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 sudo[41113]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 sudo[41116]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 sudo[41116]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 sudo[41116]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 sudo[41116]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:27.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:27.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:27.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:27.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:27.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:26 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1975915393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "307060eb-b8bb-4b80-b43e-cc22c933400f"}]: dispatch 2024-04-24T23:55:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1975915393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "307060eb-b8bb-4b80-b43e-cc22c933400f"}]': finished 2024-04-24T23:55:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:55:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:27 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/333574163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1975915393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "307060eb-b8bb-4b80-b43e-cc22c933400f"}]': finished 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:27 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/333574163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1975915393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "307060eb-b8bb-4b80-b43e-cc22c933400f"}]': finished 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:55:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:27 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/333574163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:29 smithi188 ceph-mon[33285]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:29.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:29 smithi110 ceph-mon[29187]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:29.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:29 smithi173 ceph-mon[32402]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:31.717 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:31 smithi110 ceph-mon[29187]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:31 smithi188 ceph-mon[33285]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:31.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:31 smithi173 ceph-mon[32402]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:33 smithi188 ceph-mon[33285]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:33.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:33 smithi110 ceph-mon[29187]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:33.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:33 smithi173 ceph-mon[32402]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:55:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:34.782 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:55:34.782 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:34.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:55:34.854 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:35.753 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:35 smithi173 ceph-mon[32402]: Deploying daemon osd.3 on smithi110 2024-04-24T23:55:35.753 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:35 smithi173 ceph-mon[32402]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:35 smithi188 ceph-mon[33285]: Deploying daemon osd.3 on smithi110 2024-04-24T23:55:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:35 smithi188 ceph-mon[33285]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:35.773 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:35 smithi110 ceph-mon[29187]: Deploying daemon osd.3 on smithi110 2024-04-24T23:55:35.773 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:35 smithi110 ceph-mon[29187]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:37.710 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:37 smithi110 ceph-mon[29187]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:37 smithi188 ceph-mon[33285]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:37.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:37 smithi173 ceph-mon[32402]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:39.722 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:39 smithi110 ceph-mon[29187]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:39 smithi188 ceph-mon[33285]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:39.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:39 smithi173 ceph-mon[32402]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:41.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:40 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:41.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:40 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:41.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:40 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:41.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:40 smithi188 ceph-mon[33285]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:41.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:40 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:41.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:40 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:41.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:40 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:41.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:40 smithi110 ceph-mon[29187]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:41.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:40 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:40 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:40 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:40 smithi173 ceph-mon[32402]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:43.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.609+0000 7f4b84ff9700 1 -- 172.21.15.110:0/2608092920 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4b88107f50 con 0x7f4b700a2b90 2024-04-24T23:55:43.609 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 3 on host 'smithi110' 2024-04-24T23:55:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b700a2b90 msgr2=0x7f4b700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b700a2b90 0x7f4b700a5050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b78000c00 tx=0x7f4b78005fd0 comp rx=0 tx=0).stop 2024-04-24T23:55:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b881024a0 msgr2=0x7f4b88103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b881024a0 0x7f4b88103420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000b970 tx=0x7f4b8000bd30 comp rx=0 tx=0).stop 2024-04-24T23:55:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 shutdown_connections 2024-04-24T23:55:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b700a2b90 0x7f4b700a5050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:43.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b881024a0 0x7f4b88103420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b88100900 0x7f4b88104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 --2- 172.21.15.110:0/2608092920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b88101b00 0x7f4b88102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.611+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 >> 172.21.15.110:0/2608092920 conn(0x7f4b880fbec0 msgr2=0x7f4b880ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:55:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.612+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 shutdown_connections 2024-04-24T23:55:43.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:55:43.612+0000 7f4b8dea9700 1 -- 172.21.15.110:0/2608092920 wait complete. 2024-04-24T23:55:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:43 smithi188 ceph-mon[33285]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:43.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:43 smithi110 ceph-mon[29187]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:43.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:43.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:43.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:43.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:43.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:43.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:43 smithi173 ceph-mon[32402]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:43.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:43.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:43.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:43.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:43.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:44.460 DEBUG:teuthology.orchestra.run.smithi110:osd.3> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T23:55:44.463 INFO:tasks.cephadm:Deploying osd.4 on smithi173 with /dev/vg_nvme/lv_3... 2024-04-24T23:55:44.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:55:44.708 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:55:44.753 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:44.753 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:44.753 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:44.753 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:44 smithi110 ceph-mon[29187]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:55:44.830 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:44.830 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:44.831 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:44.831 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:44 smithi173 ceph-mon[32402]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:55:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:55:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:44 smithi188 ceph-mon[33285]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:55:45.628 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:45 smithi173 ceph-mon[32402]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:45.628 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:45 smithi173 ceph-mon[32402]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:55:45.628 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:45 smithi173 ceph-mon[32402]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:55:45.628 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:45 smithi173 ceph-mon[32402]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:55:45.629 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:45 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:45.863 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:45 smithi110 ceph-mon[29187]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:45.863 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:45 smithi110 ceph-mon[29187]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:55:45.863 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:45 smithi110 ceph-mon[29187]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:55:45.863 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:45 smithi110 ceph-mon[29187]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:55:45.863 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:45 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:45 smithi188 ceph-mon[33285]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:45 smithi188 ceph-mon[33285]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:55:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:45 smithi188 ceph-mon[33285]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:55:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:45 smithi188 ceph-mon[33285]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:55:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:45 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:46.190 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 24 23:55:46 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3[50643]: 2024-04-24T23:55:46.081+0000 7fb5cfa29700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:55:46.191 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 24 23:55:46 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3[50643]: 2024-04-24T23:55:46.088+0000 7fb5c7814700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:55:46.494 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:55:46.494 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:55:46.495 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-24T23:55:46.495 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-24T23:55:46.495 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114207 s, 918 MB/s 2024-04-24T23:55:46.495 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-24T23:55:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:46 smithi188 ceph-mon[33285]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:55:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:46 smithi188 ceph-mon[33285]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:55:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:46 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:46 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:46 smithi188 ceph-mon[33285]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' 2024-04-24T23:55:46.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:46 smithi173 ceph-mon[32402]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:55:46.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:46 smithi173 ceph-mon[32402]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:55:46.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:46 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:46.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:46 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:46.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:46 smithi173 ceph-mon[32402]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' 2024-04-24T23:55:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:46 smithi110 ceph-mon[29187]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:55:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:46 smithi110 ceph-mon[29187]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:55:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:46 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:46 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:46 smithi110 ceph-mon[29187]: from='osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922]' entity='osd.3' 2024-04-24T23:55:47.192 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2024-04-24T23:55:47.437 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] boot 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] boot 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:47.822 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:47.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:47.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:47.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:47.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:47.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:47.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:55:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:55:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: osd.3 [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] boot 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:55:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:55:48.784 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:48 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi110 2024-04-24T23:55:48.784 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:48 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi110 to 8544M 2024-04-24T23:55:48.784 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:48 smithi110 ceph-mon[29187]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:55:48.828 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:48 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi110 2024-04-24T23:55:48.828 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:48 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi110 to 8544M 2024-04-24T23:55:48.828 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:48 smithi173 ceph-mon[32402]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:55:49.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 -- 172.21.15.173:0/1815968194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4102d40 msgr2=0x7f73e4105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:49.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1815968194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4102d40 0x7f73e4105130 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f73d4009990 tx=0x7f73d4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:55:49.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 -- 172.21.15.173:0/1815968194 shutdown_connections 2024-04-24T23:55:49.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1815968194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73e4107fa0 0x7f73e410a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:49.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1815968194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4102d40 0x7f73e4105130 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:49.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1815968194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4105670 0x7f73e4107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:49.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 -- 172.21.15.173:0/1815968194 >> 172.21.15.173:0/1815968194 conn(0x7f73e40fc750 msgr2=0x7f73e40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:55:49.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 -- 172.21.15.173:0/1815968194 shutdown_connections 2024-04-24T23:55:49.018 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 -- 172.21.15.173:0/1815968194 wait complete. 2024-04-24T23:55:49.018 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.017+0000 7f73ea21c700 1 Processor -- start 2024-04-24T23:55:49.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:48 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi110 2024-04-24T23:55:49.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:48 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi110 to 8544M 2024-04-24T23:55:49.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:48 smithi188 ceph-mon[33285]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:55:49.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73ea21c700 1 -- start start 2024-04-24T23:55:49.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73ea21c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4102d40 0x7f73e41026a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:55:49.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73ea21c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4105670 0x7f73e4100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:55:49.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73ea21c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73e4107fa0 0x7f73e4101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:55:49.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73ea21c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73e410dd70 con 0x7f73e4107fa0 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73ea21c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f73e410dbf0 con 0x7f73e4105670 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73ea21c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73e410def0 con 0x7f73e4102d40 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4105670 0x7f73e4100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4105670 0x7f73e4100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:53206/0 (socket says 172.21.15.173:53206) 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 -- 172.21.15.173:0/1170971283 learned_addr learned my addr 172.21.15.173:0/1170971283 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e3fff700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73e4107fa0 0x7f73e4101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e37fe700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4102d40 0x7f73e41026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 -- 172.21.15.173:0/1170971283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4102d40 msgr2=0x7f73e41026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4102d40 0x7f73e41026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 -- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73e4107fa0 msgr2=0x7f73e4101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73e4107fa0 0x7f73e4101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 -- 172.21.15.173:0/1170971283 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e4101ae0 con 0x7f73e4105670 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.018+0000 7f73e2ffd700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4105670 0x7f73e4100cf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f73d800c970 tx=0x7f73d800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:55:49.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.019+0000 7f73e0ff9700 1 -- 172.21.15.173:0/1170971283 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80078d0 con 0x7f73e4105670 2024-04-24T23:55:49.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.019+0000 7f73e0ff9700 1 -- 172.21.15.173:0/1170971283 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73d8018480 con 0x7f73e4105670 2024-04-24T23:55:49.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.019+0000 7f73e0ff9700 1 -- 172.21.15.173:0/1170971283 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d800f6b0 con 0x7f73e4105670 2024-04-24T23:55:49.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.019+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e41a2550 con 0x7f73e4105670 2024-04-24T23:55:49.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.019+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e41a2a60 con 0x7f73e4105670 2024-04-24T23:55:49.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.020+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e4068f20 con 0x7f73e4105670 2024-04-24T23:55:49.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.021+0000 7f73e0ff9700 1 -- 172.21.15.173:0/1170971283 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f73d8018630 con 0x7f73e4105670 2024-04-24T23:55:49.023 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.021+0000 7f73e0ff9700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:55:49.023 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.021+0000 7f73e0ff9700 1 -- 172.21.15.173:0/1170971283 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f73d80cded0 con 0x7f73e4105670 2024-04-24T23:55:49.023 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.021+0000 7f73e37fe700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:55:49.023 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.022+0000 7f73e37fe700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73d400b710 tx=0x7f73d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:55:49.025 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.026+0000 7f73e0ff9700 1 -- 172.21.15.173:0/1170971283 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73d809c0c0 con 0x7f73e4105670 2024-04-24T23:55:49.231 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:55:49.231+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f73e4062670 con 0x7f73cc0a2ca0 2024-04-24T23:55:49.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:49 smithi173 ceph-mon[32402]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:49.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:49 smithi173 ceph-mon[32402]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:55:49.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:55:49.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:55:49.871 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:49 smithi188 ceph-mon[33285]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:49 smithi188 ceph-mon[33285]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:55:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:55:50.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:55:50.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:49 smithi110 ceph-mon[29187]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:49 smithi110 ceph-mon[29187]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:55:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:55:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:55:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:55:50.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:50 smithi173 ceph-mon[32402]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:50.875 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:50 smithi173 ceph-mon[32402]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:55:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:50 smithi188 ceph-mon[33285]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:50 smithi188 ceph-mon[33285]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:55:51.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:50 smithi110 ceph-mon[29187]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:55:51.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:50 smithi110 ceph-mon[29187]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:55:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:51 smithi188 ceph-mon[33285]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:51 smithi188 ceph-mon[33285]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:55:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:51 smithi110 ceph-mon[29187]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:51 smithi110 ceph-mon[29187]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:55:52.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:51 smithi173 ceph-mon[32402]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:51 smithi173 ceph-mon[32402]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:55:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:52 smithi188 ceph-mon[33285]: from='client.? 172.21.15.173:0/3232419762' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]: dispatch 2024-04-24T23:55:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:52 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]: dispatch 2024-04-24T23:55:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:52 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]': finished 2024-04-24T23:55:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:52 smithi188 ceph-mon[33285]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:55:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:55:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:52 smithi188 ceph-mon[33285]: from='client.? 172.21.15.173:0/2549160478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:52 smithi110 ceph-mon[29187]: from='client.? 172.21.15.173:0/3232419762' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]: dispatch 2024-04-24T23:55:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:52 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]: dispatch 2024-04-24T23:55:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:52 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]': finished 2024-04-24T23:55:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:52 smithi110 ceph-mon[29187]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:55:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:55:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:52 smithi110 ceph-mon[29187]: from='client.? 172.21.15.173:0/2549160478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:52 smithi173 ceph-mon[32402]: from='client.? 172.21.15.173:0/3232419762' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]: dispatch 2024-04-24T23:55:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:52 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]: dispatch 2024-04-24T23:55:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:52 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12c128f9-0d7a-44ff-96c1-b2e8417851d3"}]': finished 2024-04-24T23:55:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:52 smithi173 ceph-mon[32402]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:55:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:55:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:52 smithi173 ceph-mon[32402]: from='client.? 172.21.15.173:0/2549160478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:55:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:53 smithi188 ceph-mon[33285]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:54.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:53 smithi110 ceph-mon[29187]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:54.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:53 smithi173 ceph-mon[32402]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:55:55.962 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:55 smithi173 ceph-mon[32402]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-24T23:55:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:55 smithi188 ceph-mon[33285]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-24T23:55:56.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:55 smithi110 ceph-mon[29187]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-24T23:55:57.921 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:57 smithi173 ceph-mon[32402]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-24T23:55:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:57 smithi188 ceph-mon[33285]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-24T23:55:58.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:57 smithi110 ceph-mon[29187]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-24T23:55:59.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:59 smithi173 ceph-mon[32402]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:55:59.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:55:59.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:55:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:59 smithi188 ceph-mon[33285]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:56:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:56:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:55:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:00.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:59 smithi110 ceph-mon[29187]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:56:00.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:56:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:55:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:00.705 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:00 smithi173 ceph-mon[32402]: Deploying daemon osd.4 on smithi173 2024-04-24T23:56:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:00 smithi188 ceph-mon[33285]: Deploying daemon osd.4 on smithi173 2024-04-24T23:56:01.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:00 smithi110 ceph-mon[29187]: Deploying daemon osd.4 on smithi173 2024-04-24T23:56:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:01 smithi188 ceph-mon[33285]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T23:56:02.020 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:01 smithi173 ceph-mon[32402]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T23:56:02.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:01 smithi110 ceph-mon[29187]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T23:56:03.736 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:03 smithi173 ceph-mon[32402]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-24T23:56:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:03 smithi188 ceph-mon[33285]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-24T23:56:04.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:03 smithi110 ceph-mon[29187]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-24T23:56:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:05 smithi188 ceph-mon[33285]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:56:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:06.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:05 smithi110 ceph-mon[29187]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:56:06.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:05 smithi173 ceph-mon[32402]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:56:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:07.733 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:07 smithi173 ceph-mon[32402]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:08.015 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:07 smithi188 ceph-mon[33285]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:07 smithi110 ceph-mon[29187]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:08.095 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.096+0000 7f73e0ff9700 1 -- 172.21.15.173:0/1170971283 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f73e4062670 con 0x7f73cc0a2ca0 2024-04-24T23:56:08.096 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 4 on host 'smithi173' 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f73cc0a2ca0 msgr2=0x7f73cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73d400b710 tx=0x7f73d4005fd0 comp rx=0 tx=0).stop 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4105670 msgr2=0x7f73e4100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4105670 0x7f73e4100cf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f73d800c970 tx=0x7f73d800cc80 comp rx=0 tx=0).stop 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 shutdown_connections 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f73cc0a2ca0 0x7f73cc0a5160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73e4107fa0 0x7f73e4101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4102d40 0x7f73e41026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 --2- 172.21.15.173:0/1170971283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73e4105670 0x7f73e4100cf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:08.098 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 >> 172.21.15.173:0/1170971283 conn(0x7f73e40fc750 msgr2=0x7f73e4103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:56:08.099 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 shutdown_connections 2024-04-24T23:56:08.099 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:56:08.099+0000 7f73ea21c700 1 -- 172.21.15.173:0/1170971283 wait complete. 2024-04-24T23:56:08.782 DEBUG:teuthology.orchestra.run.smithi173:osd.4> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T23:56:08.784 INFO:tasks.cephadm:Deploying osd.5 on smithi188 with /dev/vg_nvme/lv_3... 2024-04-24T23:56:08.784 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:56:09.019 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:56:09.140 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:09 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.141 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:09 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.141 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:09 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.141 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:09 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.141 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:09 smithi188 ceph-mon[33285]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:09.141 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:09 smithi188 ceph-mon[33285]: from='osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:56:09.141 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:09 smithi188 ceph-mon[33285]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:56:09.340 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:09 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.341 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:09 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.341 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:09 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.341 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:09 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.341 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:09 smithi173 ceph-mon[32402]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:09.341 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:09 smithi173 ceph-mon[32402]: from='osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:56:09.341 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:09 smithi173 ceph-mon[32402]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:56:09.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:09 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:09.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:09 smithi110 ceph-mon[29187]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:09.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:09 smithi110 ceph-mon[29187]: from='osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:56:09.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:09 smithi110 ceph-mon[29187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:56:10.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:10 smithi173 ceph-mon[32402]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:56:10.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:10 smithi173 ceph-mon[32402]: from='osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:56:10.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:10 smithi173 ceph-mon[32402]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:56:10.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:10.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:10 smithi173 ceph-mon[32402]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:56:10.602 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 24 23:56:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4[43925]: 2024-04-24T23:56:10.405+0000 7fe4bb314700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:56:10.602 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 24 23:56:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4[43925]: 2024-04-24T23:56:10.410+0000 7fe4b7107700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:56:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:10 smithi188 ceph-mon[33285]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:56:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:10 smithi188 ceph-mon[33285]: from='osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:56:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:10 smithi188 ceph-mon[33285]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:56:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:10 smithi188 ceph-mon[33285]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:56:10.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:10 smithi110 ceph-mon[29187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:56:10.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:10 smithi110 ceph-mon[29187]: from='osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:56:10.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:10 smithi110 ceph-mon[29187]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:56:10.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:10.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:10 smithi110 ceph-mon[29187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:56:10.947 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:56:10.948 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:56:10.948 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-04-24T23:56:10.948 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-04-24T23:56:10.948 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113498 s, 924 MB/s 2024-04-24T23:56:10.948 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-04-24T23:56:11.559 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:11 smithi188 ceph-mon[33285]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:56:11.560 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:11 smithi188 ceph-mon[33285]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:56:11.560 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:11.560 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:11.560 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:11 smithi188 ceph-mon[33285]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:11.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_3 2024-04-24T23:56:11.693 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:11 smithi173 ceph-mon[32402]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:56:11.693 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:11 smithi173 ceph-mon[32402]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:56:11.694 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:11.694 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:11.694 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:11 smithi173 ceph-mon[32402]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:11 smithi110 ceph-mon[29187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:56:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:11 smithi110 ceph-mon[29187]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:56:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:11 smithi110 ceph-mon[29187]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:56:11.820 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:56:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:56:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:56:12.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] boot 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:12.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:12.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] boot 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:12.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:12.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:12.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:12.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:12.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:12.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:56:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:56:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: osd.4 [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] boot 2024-04-24T23:56:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:56:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:56:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 -- 172.21.15.188:0/3296075301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 msgr2=0x7f40fc109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 --2- 172.21.15.188:0/3296075301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 0x7f40fc109650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f40f400b0a0 tx=0x7f40f400b3b0 comp rx=0 tx=0).stop 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 -- 172.21.15.188:0/3296075301 shutdown_connections 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 --2- 172.21.15.188:0/3296075301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 0x7f40fc109650 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 --2- 172.21.15.188:0/3296075301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40fc102790 0x7f40fc102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 --2- 172.21.15.188:0/3296075301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1013f0 0x7f40fc101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 -- 172.21.15.188:0/3296075301 >> 172.21.15.188:0/3296075301 conn(0x7f40fc0fbec0 msgr2=0x7f40fc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 -- 172.21.15.188:0/3296075301 shutdown_connections 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 -- 172.21.15.188:0/3296075301 wait complete. 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.416+0000 7f4103ac9700 1 Processor -- start 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4103ac9700 1 -- start start 2024-04-24T23:56:13.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4103ac9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1013f0 0x7f40fc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4103ac9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 0x7f40fc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4103ac9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40fc102790 0x7f40fc1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4103ac9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc10cc00 con 0x7f40fc102790 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4103ac9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc10ca80 con 0x7f40fc1013f0 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4103ac9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc10cd80 con 0x7f40fc101db0 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 0x7f40fc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 0x7f40fc19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:58022/0 (socket says 172.21.15.188:58022) 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 -- 172.21.15.188:0/2748747904 learned_addr learned my addr 172.21.15.188:0/2748747904 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4102066700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40fc102790 0x7f40fc1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101865700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1013f0 0x7f40fc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 -- 172.21.15.188:0/2748747904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1013f0 msgr2=0x7f40fc19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1013f0 0x7f40fc19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:13.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 -- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40fc102790 msgr2=0x7f40fc1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40fc102790 0x7f40fc1a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 -- 172.21.15.188:0/2748747904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc1a2170 con 0x7f40fc101db0 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4102066700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40fc102790 0x7f40fc1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.417+0000 7f4101064700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 0x7f40fc19aa50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00d980 tx=0x7f40ec00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.418+0000 7f40f2ffd700 1 -- 172.21.15.188:0/2748747904 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec016070 con 0x7f40fc101db0 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.418+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc1a2460 con 0x7f40fc101db0 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.418+0000 7f40f2ffd700 1 -- 172.21.15.188:0/2748747904 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40ec009d80 con 0x7f40fc101db0 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.418+0000 7f40f2ffd700 1 -- 172.21.15.188:0/2748747904 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00f3c0 con 0x7f40fc101db0 2024-04-24T23:56:13.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.418+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc1a29f0 con 0x7f40fc101db0 2024-04-24T23:56:13.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.419+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40fc068f20 con 0x7f40fc101db0 2024-04-24T23:56:13.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.420+0000 7f40f2ffd700 1 -- 172.21.15.188:0/2748747904 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f40ec019070 con 0x7f40fc101db0 2024-04-24T23:56:13.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.421+0000 7f40f2ffd700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2c20 0x7f40e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:56:13.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.421+0000 7f40f2ffd700 1 -- 172.21.15.188:0/2748747904 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f40ec0cda80 con 0x7f40fc101db0 2024-04-24T23:56:13.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.421+0000 7f4101865700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2c20 0x7f40e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:56:13.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.421+0000 7f4101865700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2c20 0x7f40e80a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f40fc069cf0 tx=0x7f40f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:56:13.428 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.429+0000 7f40f2ffd700 1 -- 172.21.15.188:0/2748747904 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f40ec09ba50 con 0x7f40fc101db0 2024-04-24T23:56:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:13 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi173 2024-04-24T23:56:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:13 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-24T23:56:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:13 smithi188 ceph-mon[33285]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:56:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:13 smithi188 ceph-mon[33285]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:13.636 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:13.636+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f40fc0fe040 con 0x7f40e80a2c20 2024-04-24T23:56:13.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:13 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi173 2024-04-24T23:56:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:13 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-24T23:56:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:13 smithi110 ceph-mon[29187]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:56:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:13 smithi110 ceph-mon[29187]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:13.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:13 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi173 2024-04-24T23:56:13.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:13 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-24T23:56:13.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:13 smithi173 ceph-mon[32402]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:56:13.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:13 smithi173 ceph-mon[32402]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:14 smithi188 ceph-mon[33285]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:56:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:56:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:56:14.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:14.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:14 smithi110 ceph-mon[29187]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:56:14.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:56:14.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:56:14.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:14.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:14 smithi173 ceph-mon[32402]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:56:14.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:56:14.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:56:14.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:15.488 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:15 smithi188 ceph-mon[33285]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:15.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:15 smithi110 ceph-mon[29187]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:15.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:15 smithi173 ceph-mon[32402]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:16.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:16 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/40172532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]: dispatch 2024-04-24T23:56:16.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:16 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]: dispatch 2024-04-24T23:56:16.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:16 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]': finished 2024-04-24T23:56:16.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:16 smithi188 ceph-mon[33285]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:56:16.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:16 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:16.696 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:16 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/40172532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]: dispatch 2024-04-24T23:56:16.696 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:16 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]: dispatch 2024-04-24T23:56:16.696 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:16 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]': finished 2024-04-24T23:56:16.696 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:16 smithi173 ceph-mon[32402]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:56:16.697 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:16 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:16.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:16 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/40172532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]: dispatch 2024-04-24T23:56:16.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:16 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]: dispatch 2024-04-24T23:56:16.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:16 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b479fe2-2120-4b2c-a0d6-986d5d7582d0"}]': finished 2024-04-24T23:56:16.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:16 smithi110 ceph-mon[29187]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:56:16.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:16 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:17.687 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:17 smithi110 ceph-mon[29187]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:17.687 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:17 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/3740196648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:56:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:17 smithi188 ceph-mon[33285]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:17 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/3740196648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:56:17.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:17 smithi173 ceph-mon[32402]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:17 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/3740196648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:56:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:19 smithi188 ceph-mon[33285]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:19.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:19 smithi110 ceph-mon[29187]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:19.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:19 smithi173 ceph-mon[32402]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:21.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:21 smithi110 ceph-mon[29187]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:21.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:21 smithi173 ceph-mon[32402]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:21.891 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:21 smithi188 ceph-mon[33285]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:23.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:23 smithi110 ceph-mon[29187]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:23.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:23 smithi173 ceph-mon[32402]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:23 smithi188 ceph-mon[33285]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:24.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:24 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:56:24.614 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:24 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:24.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:24 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:56:24.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:24 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:24.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:24 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:56:24.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:24 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:25.672 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:25 smithi188 ceph-mon[33285]: Deploying daemon osd.5 on smithi188 2024-04-24T23:56:25.672 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:25 smithi188 ceph-mon[33285]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:25.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:25 smithi110 ceph-mon[29187]: Deploying daemon osd.5 on smithi188 2024-04-24T23:56:25.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:25 smithi110 ceph-mon[29187]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:25.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:25 smithi173 ceph-mon[32402]: Deploying daemon osd.5 on smithi188 2024-04-24T23:56:25.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:25 smithi173 ceph-mon[32402]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:27.742 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:27 smithi188 ceph-mon[33285]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:27 smithi110 ceph-mon[29187]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:27.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:27 smithi173 ceph-mon[32402]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:29.751 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:29 smithi188 ceph-mon[33285]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:29.751 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:29.751 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:29.751 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:29.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:29 smithi110 ceph-mon[29187]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:29.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:29.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:29.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:29.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:29 smithi173 ceph-mon[32402]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:31.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:31 smithi110 ceph-mon[29187]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:31.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:31 smithi173 ceph-mon[32402]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:32.006 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:31 smithi188 ceph-mon[33285]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:32.701 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 5 on host 'smithi188' 2024-04-24T23:56:32.701 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.701+0000 7f40f2ffd700 1 -- 172.21.15.188:0/2748747904 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f40fc0fe040 con 0x7f40e80a2c20 2024-04-24T23:56:32.703 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2c20 msgr2=0x7f40e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:32.703 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2c20 0x7f40e80a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f40fc069cf0 tx=0x7f40f8009260 comp rx=0 tx=0).stop 2024-04-24T23:56:32.703 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 msgr2=0x7f40fc19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:32.703 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 0x7f40fc19aa50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00d980 tx=0x7f40ec00dc90 comp rx=0 tx=0).stop 2024-04-24T23:56:32.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 shutdown_connections 2024-04-24T23:56:32.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2c20 0x7f40e80a50e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:32.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40fc102790 0x7f40fc1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:32.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1013f0 0x7f40fc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:32.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 --2- 172.21.15.188:0/2748747904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40fc101db0 0x7f40fc19aa50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:32.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 >> 172.21.15.188:0/2748747904 conn(0x7f40fc0fbec0 msgr2=0x7f40fc0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:56:32.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.703+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 shutdown_connections 2024-04-24T23:56:32.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:56:32.704+0000 7f4103ac9700 1 -- 172.21.15.188:0/2748747904 wait complete. 2024-04-24T23:56:33.329 DEBUG:teuthology.orchestra.run.smithi188:osd.5> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T23:56:33.331 INFO:tasks.cephadm:Deploying osd.6 on smithi110 with /dev/vg_nvme/lv_2... 2024-04-24T23:56:33.331 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:56:33.581 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:56:33.703 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:33 smithi110 ceph-mon[29187]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:33.703 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.703 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.703 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:33 smithi110 ceph-mon[29187]: from='osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:56:33.704 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:33 smithi110 ceph-mon[29187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:56:33.704 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.704 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:33 smithi188 ceph-mon[33285]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:33 smithi188 ceph-mon[33285]: from='osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:56:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:33 smithi188 ceph-mon[33285]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:56:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:33 smithi173 ceph-mon[32402]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:33.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:33 smithi173 ceph-mon[32402]: from='osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:56:33.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:33 smithi173 ceph-mon[32402]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:56:33.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:33.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:34.986 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:34 smithi110 ceph-mon[29187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:56:34.986 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:34 smithi110 ceph-mon[29187]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:56:34.986 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:34.986 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:34 smithi110 ceph-mon[29187]: from='osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:56:34.986 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:34 smithi110 ceph-mon[29187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:56:34.986 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:34 smithi110 ceph-mon[29187]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:34 smithi188 ceph-mon[33285]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:56:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:34 smithi188 ceph-mon[33285]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:56:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:34 smithi188 ceph-mon[33285]: from='osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:56:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:34 smithi188 ceph-mon[33285]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:56:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:34 smithi188 ceph-mon[33285]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:35.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 24 23:56:34 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-24T23:56:34.705+0000 7f9d75762700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:56:35.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 24 23:56:34 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-24T23:56:34.712+0000 7f9d71555700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:56:35.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:34 smithi173 ceph-mon[32402]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:56:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:34 smithi173 ceph-mon[32402]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:56:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:34 smithi173 ceph-mon[32402]: from='osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:56:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:34 smithi173 ceph-mon[32402]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:56:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:34 smithi173 ceph-mon[32402]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:35.639 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:56:35.639 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:56:35.639 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-04-24T23:56:35.640 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-04-24T23:56:35.640 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00879546 s, 1.2 GB/s 2024-04-24T23:56:35.640 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 2024-04-24T23:56:35.640 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-04-24T23:56:36.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:35 smithi173 ceph-mon[32402]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:56:36.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:35 smithi173 ceph-mon[32402]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:56:36.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:36.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:35 smithi188 ceph-mon[33285]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:56:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:35 smithi188 ceph-mon[33285]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:56:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:36.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:35 smithi110 ceph-mon[29187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:56:36.026 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:35 smithi110 ceph-mon[29187]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:56:36.026 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:36.026 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:36.373 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-04-24T23:56:36.623 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:56:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:36 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:56:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:36 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:56:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:36 smithi188 ceph-mon[33285]: osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] boot 2024-04-24T23:56:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:36 smithi188 ceph-mon[33285]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T23:56:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:36 smithi188 ceph-mon[33285]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:37.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:36 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:56:37.103 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:36 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:56:37.104 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:36 smithi110 ceph-mon[29187]: osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] boot 2024-04-24T23:56:37.104 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:36 smithi110 ceph-mon[29187]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T23:56:37.104 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:37.104 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:36 smithi110 ceph-mon[29187]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:37.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:36 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:56:37.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:36 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:56:37.105 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:36 smithi173 ceph-mon[32402]: osd.5 [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] boot 2024-04-24T23:56:37.105 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:36 smithi173 ceph-mon[32402]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T23:56:37.105 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:56:37.105 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:36 smithi173 ceph-mon[32402]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi188 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi188 to 8544M 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:37.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:37.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:37.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:37.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:37.989 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:37 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi188 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi188 to 8544M 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:38.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:38.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:38.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:37 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:56:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi188 2024-04-24T23:56:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi188 to 8544M 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:38.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:37 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.417+0000 7f35d68bd700 1 -- 172.21.15.110:0/613751957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 msgr2=0x7f35d01096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.417+0000 7f35d68bd700 1 --2- 172.21.15.110:0/613751957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 0x7f35d01096b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f35c800b0d0 tx=0x7f35c800b3e0 comp rx=0 tx=0).stop 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 -- 172.21.15.110:0/613751957 shutdown_connections 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 --2- 172.21.15.110:0/613751957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 0x7f35d01096b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 --2- 172.21.15.110:0/613751957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0102710 0x7f35d0102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 --2- 172.21.15.110:0/613751957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d0101450 0x7f35d01018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 -- 172.21.15.110:0/613751957 >> 172.21.15.110:0/613751957 conn(0x7f35d00fbec0 msgr2=0x7f35d00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 -- 172.21.15.110:0/613751957 shutdown_connections 2024-04-24T23:56:38.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 -- 172.21.15.110:0/613751957 wait complete. 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 Processor -- start 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 -- start start 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.418+0000 7f35d68bd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d0101450 0x7f35d01973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35d68bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 0x7f35d0197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35d68bd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0102710 0x7f35d019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35d68bd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010cd10 con 0x7f35d0101df0 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35d68bd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010cb90 con 0x7f35d0102710 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35d68bd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010ce90 con 0x7f35d0101450 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 0x7f35d0197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 0x7f35d0197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58902/0 (socket says 172.21.15.110:58902) 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 -- 172.21.15.110:0/894146221 learned_addr learned my addr 172.21.15.110:0/894146221 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35d4e5a700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0102710 0x7f35d019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:56:38.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cffff700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d0101450 0x7f35d01973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 -- 172.21.15.110:0/894146221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d0101450 msgr2=0x7f35d01973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d0101450 0x7f35d01973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 -- 172.21.15.110:0/894146221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0102710 msgr2=0x7f35d019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0102710 0x7f35d019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 -- 172.21.15.110:0/894146221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d019f040 con 0x7f35d0101df0 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cffff700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d0101450 0x7f35d01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cf7fe700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 0x7f35d0197920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f35c400da10 tx=0x7f35c400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cd7fa700 1 -- 172.21.15.110:0/894146221 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c401b070 con 0x7f35d0101df0 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d01003a0 con 0x7f35d0101df0 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cd7fa700 1 -- 172.21.15.110:0/894146221 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35c400ec50 con 0x7f35d0101df0 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35cd7fa700 1 -- 172.21.15.110:0/894146221 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40145a0 con 0x7f35d0101df0 2024-04-24T23:56:38.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.419+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d0100960 con 0x7f35d0101df0 2024-04-24T23:56:38.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.420+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d0068f20 con 0x7f35d0101df0 2024-04-24T23:56:38.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.421+0000 7f35cd7fa700 1 -- 172.21.15.110:0/894146221 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f35c401e070 con 0x7f35d0101df0 2024-04-24T23:56:38.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.422+0000 7f35cd7fa700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35b80a2c20 0x7f35b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:56:38.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.422+0000 7f35cffff700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35b80a2c20 0x7f35b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:56:38.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.422+0000 7f35cd7fa700 1 -- 172.21.15.110:0/894146221 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f35c40cf710 con 0x7f35d0101df0 2024-04-24T23:56:38.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.423+0000 7f35cffff700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35b80a2c20 0x7f35b80a50e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f35d0069c90 tx=0x7f35c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:56:38.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.427+0000 7f35cd7fa700 1 -- 172.21.15.110:0/894146221 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f35c409d510 con 0x7f35d0101df0 2024-04-24T23:56:38.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:38.637+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f35d0100c20 con 0x7f35b80a2c20 2024-04-24T23:56:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:38 smithi188 ceph-mon[33285]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:56:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:38 smithi188 ceph-mon[33285]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:56:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:38 smithi188 ceph-mon[33285]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:38 smithi188 ceph-mon[33285]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:56:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:56:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:56:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:39.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:38 smithi110 ceph-mon[29187]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:56:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:38 smithi110 ceph-mon[29187]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:56:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:38 smithi110 ceph-mon[29187]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:38 smithi110 ceph-mon[29187]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:56:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:56:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:56:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:38 smithi173 ceph-mon[32402]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:56:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:38 smithi173 ceph-mon[32402]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:56:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:38 smithi173 ceph-mon[32402]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:38 smithi173 ceph-mon[32402]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:56:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:56:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:56:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:40.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:40 smithi110 ceph-mon[29187]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:56:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:40 smithi188 ceph-mon[33285]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:56:40.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:40 smithi173 ceph-mon[32402]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:56:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:41 smithi188 ceph-mon[33285]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:41 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3296070859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf7fd3c-3d98-411e-b50b-bbe5ea52f664"}]: dispatch 2024-04-24T23:56:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:41 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3296070859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bf7fd3c-3d98-411e-b50b-bbe5ea52f664"}]': finished 2024-04-24T23:56:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:41 smithi188 ceph-mon[33285]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T23:56:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:56:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:41 smithi110 ceph-mon[29187]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:41 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3296070859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf7fd3c-3d98-411e-b50b-bbe5ea52f664"}]: dispatch 2024-04-24T23:56:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:41 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3296070859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bf7fd3c-3d98-411e-b50b-bbe5ea52f664"}]': finished 2024-04-24T23:56:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:41 smithi110 ceph-mon[29187]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T23:56:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:56:41.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:41 smithi173 ceph-mon[32402]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:41.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:41 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3296070859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bf7fd3c-3d98-411e-b50b-bbe5ea52f664"}]: dispatch 2024-04-24T23:56:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:41 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3296070859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bf7fd3c-3d98-411e-b50b-bbe5ea52f664"}]': finished 2024-04-24T23:56:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:41 smithi173 ceph-mon[32402]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T23:56:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:56:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:42 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3867021617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:56:42.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:42 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3867021617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:56:42.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:42 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3867021617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:56:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:43 smithi188 ceph-mon[33285]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:43.785 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:43 smithi110 ceph-mon[29187]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:43.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:43 smithi173 ceph-mon[32402]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:45.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:45 smithi110 ceph-mon[29187]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:56:45.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:45 smithi173 ceph-mon[32402]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:56:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:45 smithi188 ceph-mon[33285]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:56:47.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:47 smithi173 ceph-mon[32402]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:56:47.605 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:47 smithi110 ceph-mon[29187]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:56:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:47 smithi188 ceph-mon[33285]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:56:49.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:49 smithi110 ceph-mon[29187]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-24T23:56:49.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:56:49.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:49.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:49 smithi173 ceph-mon[32402]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-24T23:56:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:56:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:49 smithi188 ceph-mon[33285]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-24T23:56:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:56:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:50.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:50 smithi110 ceph-mon[29187]: Deploying daemon osd.6 on smithi110 2024-04-24T23:56:50.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:50 smithi173 ceph-mon[32402]: Deploying daemon osd.6 on smithi110 2024-04-24T23:56:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:50 smithi188 ceph-mon[33285]: Deploying daemon osd.6 on smithi110 2024-04-24T23:56:51.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:51 smithi110 ceph-mon[29187]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:56:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:51 smithi173 ceph-mon[32402]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:56:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:51 smithi188 ceph-mon[33285]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:56:53.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:53 smithi110 ceph-mon[29187]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-24T23:56:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:53 smithi173 ceph-mon[32402]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-24T23:56:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:53 smithi188 ceph-mon[33285]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-24T23:56:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:55 smithi110 ceph-mon[29187]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:56:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:55.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:55 smithi173 ceph-mon[32402]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:56:55.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:55 smithi188 ceph-mon[33285]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:56:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:56:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:57 smithi110 ceph-mon[29187]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:57.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:57 smithi173 ceph-mon[32402]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:57 smithi188 ceph-mon[33285]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:59.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.617+0000 7f35cd7fa700 1 -- 172.21.15.110:0/894146221 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f35d0100c20 con 0x7f35b80a2c20 2024-04-24T23:56:59.618 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 6 on host 'smithi110' 2024-04-24T23:56:59.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.619+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35b80a2c20 msgr2=0x7f35b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.619+0000 7f35d68bd700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35b80a2c20 0x7f35b80a50e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f35d0069c90 tx=0x7f35c0008040 comp rx=0 tx=0).stop 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.619+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 msgr2=0x7f35d0197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.619+0000 7f35d68bd700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 0x7f35d0197920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f35c400da10 tx=0x7f35c400dd20 comp rx=0 tx=0).stop 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.620+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 shutdown_connections 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.620+0000 7f35d68bd700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35b80a2c20 0x7f35b80a50e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.620+0000 7f35d68bd700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0102710 0x7f35d019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.620+0000 7f35d68bd700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d0101450 0x7f35d01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.620+0000 7f35d68bd700 1 --2- 172.21.15.110:0/894146221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35d0101df0 0x7f35d0197920 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.620+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 >> 172.21.15.110:0/894146221 conn(0x7f35d00fbec0 msgr2=0x7f35d0105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.620+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 shutdown_connections 2024-04-24T23:56:59.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:56:59.620+0000 7f35d68bd700 1 -- 172.21.15.110:0/894146221 wait complete. 2024-04-24T23:56:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:59 smithi173 ceph-mon[32402]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:59 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:59.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:56:59 smithi173 ceph-mon[32402]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:56:59.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:59 smithi110 ceph-mon[29187]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:56:59.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:59.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:59.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:56:59.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:56:59.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:59 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:56:59.945 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:56:59 smithi110 ceph-mon[29187]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:57:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:59 smithi188 ceph-mon[33285]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:57:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:59 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:56:59 smithi188 ceph-mon[33285]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:57:00.304 DEBUG:teuthology.orchestra.run.smithi110:osd.6> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T23:57:00.307 INFO:tasks.cephadm:Deploying osd.7 on smithi173 with /dev/vg_nvme/lv_2... 2024-04-24T23:57:00.307 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:57:00.552 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:57:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:00 smithi173 ceph-mon[32402]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:57:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:00 smithi173 ceph-mon[32402]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:57:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:00 smithi173 ceph-mon[32402]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:57:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:00 smithi188 ceph-mon[33285]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:57:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:00 smithi188 ceph-mon[33285]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:57:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:00 smithi188 ceph-mon[33285]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:57:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:01.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:01.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-mon[29187]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:57:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-mon[29187]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:57:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-mon[29187]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:57:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:01.036 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-24T23:57:00.604+0000 7fe5de60b700 -1 osd.6 0 waiting for initial osdmap 2024-04-24T23:57:01.037 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 24 23:57:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-24T23:57:00.612+0000 7fe5da3fe700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:57:01.914 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:01 smithi110 ceph-mon[29187]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:57:01.914 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:01 smithi110 ceph-mon[29187]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:57:01.915 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:01 smithi110 ceph-mon[29187]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:57:01.915 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:01 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:01.915 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:01 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:01.920 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:01 smithi173 ceph-mon[32402]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:57:01.920 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:01 smithi173 ceph-mon[32402]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:57:01.920 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:01 smithi173 ceph-mon[32402]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:57:01.920 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:01 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:01.920 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:01 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:01 smithi188 ceph-mon[33285]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:57:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:01 smithi188 ceph-mon[33285]: from='osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:57:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:01 smithi188 ceph-mon[33285]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:57:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:01 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:01 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:02.667 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:57:02.667 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:57:02.667 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-24T23:57:02.667 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-24T23:57:02.667 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113487 s, 924 MB/s 2024-04-24T23:57:02.667 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-24T23:57:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:02 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:57:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:02 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:57:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:02 smithi173 ceph-mon[32402]: osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] boot 2024-04-24T23:57:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:02 smithi173 ceph-mon[32402]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T23:57:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:02 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:57:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:02 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:57:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:02 smithi188 ceph-mon[33285]: osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] boot 2024-04-24T23:57:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:02 smithi188 ceph-mon[33285]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T23:57:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:03.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:02 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:57:03.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:02 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:57:03.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:03.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:02 smithi110 ceph-mon[29187]: osd.6 [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] boot 2024-04-24T23:57:03.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:02 smithi110 ceph-mon[29187]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T23:57:03.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:57:03.347 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2024-04-24T23:57:03.588 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:03.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:03.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:03 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:03.965 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:03.966 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:03 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:57:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:57:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:03 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:04.669 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:04 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi110 2024-04-24T23:57:04.669 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:04 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi110 to 5696M 2024-04-24T23:57:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:04 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi110 2024-04-24T23:57:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:04 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi110 to 5696M 2024-04-24T23:57:05.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:04 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi110 2024-04-24T23:57:05.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:04 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi110 to 5696M 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 -- 172.21.15.173:0/1656739533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 msgr2=0x7f034c1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 --2- 172.21.15.173:0/1656739533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 0x7f034c1095e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f034400b0d0 tx=0x7f034400b3e0 comp rx=0 tx=0).stop 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 -- 172.21.15.173:0/1656739533 shutdown_connections 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 --2- 172.21.15.173:0/1656739533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 0x7f034c1095e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 --2- 172.21.15.173:0/1656739533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f034c101190 0x7f034c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 --2- 172.21.15.173:0/1656739533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f034c102390 0x7f034c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 -- 172.21.15.173:0/1656739533 >> 172.21.15.173:0/1656739533 conn(0x7f034c0fc730 msgr2=0x7f034c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 -- 172.21.15.173:0/1656739533 shutdown_connections 2024-04-24T23:57:05.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.209+0000 7f0353158700 1 -- 172.21.15.173:0/1656739533 wait complete. 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0353158700 1 Processor -- start 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0353158700 1 -- start start 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0353158700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f034c101190 0x7f034c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0353158700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f034c102390 0x7f034c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0353158700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 0x7f034c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0353158700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f034c10cc10 con 0x7f034c101190 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0353158700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f034c10ca90 con 0x7f034c102d30 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0353158700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f034c10cd90 con 0x7f034c102390 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 0x7f034c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 0x7f034c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:47430/0 (socket says 172.21.15.173:47430) 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 -- 172.21.15.173:0/770727422 learned_addr learned my addr 172.21.15.173:0/770727422 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0343fff700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f034c102390 0x7f034c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0350ef4700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f034c101190 0x7f034c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:05.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 -- 172.21.15.173:0/770727422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f034c102390 msgr2=0x7f034c1976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f034c102390 0x7f034c1976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 -- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f034c101190 msgr2=0x7f034c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f034c101190 0x7f034c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 -- 172.21.15.173:0/770727422 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c19edf0 con 0x7f034c102d30 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f0343fff700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f034c102390 0x7f034c1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.210+0000 7f03516f5700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 0x7f034c19e750 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0344009fd0 tx=0x7f0344004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.211+0000 7f0341ffb700 1 -- 172.21.15.173:0/770727422 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034401f030 con 0x7f034c102d30 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.211+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c19f080 con 0x7f034c102d30 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.211+0000 7f0341ffb700 1 -- 172.21.15.173:0/770727422 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0344007bb0 con 0x7f034c102d30 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.211+0000 7f0341ffb700 1 -- 172.21.15.173:0/770727422 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344004850 con 0x7f034c102d30 2024-04-24T23:57:05.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.211+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c19f4d0 con 0x7f034c102d30 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.212+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c068f20 con 0x7f034c102d30 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.213+0000 7f0341ffb700 1 -- 172.21.15.173:0/770727422 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0344003fd0 con 0x7f034c102d30 2024-04-24T23:57:05.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.213+0000 7f0341ffb700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f03340a2c20 0x7f03340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.214+0000 7f0341ffb700 1 -- 172.21.15.173:0/770727422 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f034400f020 con 0x7f034c102d30 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.214+0000 7f0350ef4700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f03340a2c20 0x7f03340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:05.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.214+0000 7f0350ef4700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f03340a2c20 0x7f03340a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0348005fd0 tx=0x7f0348005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:05.218 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.218+0000 7f0341ffb700 1 -- 172.21.15.173:0/770727422 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0344029020 con 0x7f034c102d30 2024-04-24T23:57:05.422 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:05.422+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f034c062670 con 0x7f03340a2c20 2024-04-24T23:57:05.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:05 smithi188 ceph-mon[33285]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:05 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:05 smithi110 ceph-mon[29187]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:05 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:06.041 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:05 smithi173 ceph-mon[32402]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:06.041 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:06.042 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:06.042 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:05 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:06 smithi188 ceph-mon[33285]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:07.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:06 smithi110 ceph-mon[29187]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:07.056 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:06 smithi173 ceph-mon[32402]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:07 smithi188 ceph-mon[33285]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:08.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:07 smithi110 ceph-mon[29187]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:08.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:07 smithi173 ceph-mon[32402]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:08.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:08 smithi173 ceph-mon[32402]: from='client.? 172.21.15.173:0/3124156073' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]: dispatch 2024-04-24T23:57:08.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:08 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]: dispatch 2024-04-24T23:57:08.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:08 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]': finished 2024-04-24T23:57:08.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:08 smithi173 ceph-mon[32402]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T23:57:08.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:08 smithi188 ceph-mon[33285]: from='client.? 172.21.15.173:0/3124156073' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]: dispatch 2024-04-24T23:57:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:08 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]: dispatch 2024-04-24T23:57:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:08 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]': finished 2024-04-24T23:57:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:08 smithi188 ceph-mon[33285]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T23:57:09.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:08 smithi110 ceph-mon[29187]: from='client.? 172.21.15.173:0/3124156073' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]: dispatch 2024-04-24T23:57:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:08 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]: dispatch 2024-04-24T23:57:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:08 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19d1620c-d00e-4c4a-b0a3-acf84b3021ec"}]': finished 2024-04-24T23:57:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:08 smithi110 ceph-mon[29187]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T23:57:09.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:09 smithi188 ceph-mon[33285]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:09 smithi188 ceph-mon[33285]: from='client.? 172.21.15.173:0/1127852830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:57:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:09 smithi110 ceph-mon[29187]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:09 smithi110 ceph-mon[29187]: from='client.? 172.21.15.173:0/1127852830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:57:10.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:09 smithi173 ceph-mon[32402]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:09 smithi173 ceph-mon[32402]: from='client.? 172.21.15.173:0/1127852830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:57:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:11 smithi188 ceph-mon[33285]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:12.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:11 smithi110 ceph-mon[29187]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:12.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:11 smithi173 ceph-mon[32402]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:13.849 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:13 smithi173 ceph-mon[32402]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:13 smithi188 ceph-mon[33285]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:14.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:13 smithi110 ceph-mon[29187]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:15 smithi188 ceph-mon[33285]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:15 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:57:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:15 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:16.020 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:15 smithi173 ceph-mon[32402]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:16.020 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:15 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:57:16.020 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:15 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:15 smithi110 ceph-mon[29187]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:15 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:57:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:15 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:16 smithi188 ceph-mon[33285]: Deploying daemon osd.7 on smithi173 2024-04-24T23:57:17.025 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:16 smithi173 ceph-mon[32402]: Deploying daemon osd.7 on smithi173 2024-04-24T23:57:17.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:16 smithi110 ceph-mon[29187]: Deploying daemon osd.7 on smithi173 2024-04-24T23:57:17.936 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:17 smithi110 ceph-mon[29187]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:18.015 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:17 smithi173 ceph-mon[32402]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:17 smithi188 ceph-mon[33285]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:19 smithi188 ceph-mon[33285]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:19 smithi110 ceph-mon[29187]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:20.045 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:19 smithi173 ceph-mon[32402]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:21.942 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:21 smithi173 ceph-mon[32402]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:21.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:21.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:21.943 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:21 smithi188 ceph-mon[33285]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:21 smithi110 ceph-mon[29187]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:23.927 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:23 smithi173 ceph-mon[32402]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:23 smithi188 ceph-mon[33285]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:23 smithi110 ceph-mon[29187]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:24.986 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.986+0000 7f0341ffb700 1 -- 172.21.15.173:0/770727422 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f034c062670 con 0x7f03340a2c20 2024-04-24T23:57:24.987 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 7 on host 'smithi173' 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f03340a2c20 msgr2=0x7f03340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f03340a2c20 0x7f03340a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0348005fd0 tx=0x7f0348005dd0 comp rx=0 tx=0).stop 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 msgr2=0x7f034c19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 0x7f034c19e750 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0344009fd0 tx=0x7f0344004ef0 comp rx=0 tx=0).stop 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 shutdown_connections 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f03340a2c20 0x7f03340a50e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f034c102d30 0x7f034c19e750 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f034c101190 0x7f034c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:24.989 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 --2- 172.21.15.173:0/770727422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f034c102390 0x7f034c1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:24.990 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 >> 172.21.15.173:0/770727422 conn(0x7f034c0fc730 msgr2=0x7f034c105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:24.990 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 shutdown_connections 2024-04-24T23:57:24.990 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:57:24.989+0000 7f0353158700 1 -- 172.21.15.173:0/770727422 wait complete. 2024-04-24T23:57:25.304 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.304 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.304 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:25.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:25.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:25.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:57:25.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:57:25.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:25.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:25.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:25.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:25.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:57:25.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:57:25.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:25.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:25.650 DEBUG:teuthology.orchestra.run.smithi173:osd.7> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T23:57:25.652 INFO:tasks.cephadm:Deploying osd.8 on smithi188 with /dev/vg_nvme/lv_2... 2024-04-24T23:57:25.652 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:57:25.897 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:57:26.314 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:26 smithi188 ceph-mon[33285]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:57:26.314 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:26 smithi188 ceph-mon[33285]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:57:26.314 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:26.315 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:26 smithi188 ceph-mon[33285]: from='osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:57:26.315 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:26 smithi188 ceph-mon[33285]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:57:26.351 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:26 smithi173 ceph-mon[32402]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:57:26.351 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:26 smithi173 ceph-mon[32402]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:57:26.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:26.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:26 smithi173 ceph-mon[32402]: from='osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:57:26.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:26 smithi173 ceph-mon[32402]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:57:26.352 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 24 23:57:26 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-24T23:57:26.134+0000 7fe07ec71700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T23:57:26.352 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 24 23:57:26 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-24T23:57:26.142+0000 7fe07aa64700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:57:26.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:26 smithi110 ceph-mon[29187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:57:26.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:26 smithi110 ceph-mon[29187]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:57:26.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:26.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:26 smithi110 ceph-mon[29187]: from='osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:57:26.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:26 smithi110 ceph-mon[29187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:57:27.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:27 smithi188 ceph-mon[33285]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:57:27.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:27 smithi188 ceph-mon[33285]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:57:27.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:27.383 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:27.384 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:27 smithi188 ceph-mon[33285]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:27.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:27 smithi110 ceph-mon[29187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:57:27.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:27 smithi110 ceph-mon[29187]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:57:27.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:27.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:27.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:27 smithi110 ceph-mon[29187]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:27.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:27 smithi173 ceph-mon[32402]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:57:27.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:27 smithi173 ceph-mon[32402]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:57:27.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:27.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:27.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:27 smithi173 ceph-mon[32402]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:57:27.951 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:57:27.951 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:57:27.951 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-04-24T23:57:27.951 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-04-24T23:57:27.951 INFO:teuthology.orchestra.run.smithi188.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111453 s, 941 MB/s 2024-04-24T23:57:27.951 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 2024-04-24T23:57:27.951 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-04-24T23:57:28.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:28 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:57:28.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:28 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:57:28.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:28 smithi173 ceph-mon[32402]: osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] boot 2024-04-24T23:57:28.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:28 smithi173 ceph-mon[32402]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T23:57:28.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:28 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:28.305 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:28 smithi173 ceph-mon[32402]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:57:28.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:28 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:57:28.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:28 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:57:28.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:28 smithi188 ceph-mon[33285]: osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] boot 2024-04-24T23:57:28.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:28 smithi188 ceph-mon[33285]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T23:57:28.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:28 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:28.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:28 smithi188 ceph-mon[33285]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:57:28.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:28 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:57:28.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:28 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:57:28.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:28 smithi110 ceph-mon[29187]: osd.7 [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] boot 2024-04-24T23:57:28.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:28 smithi110 ceph-mon[29187]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T23:57:28.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:57:28.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:28 smithi110 ceph-mon[29187]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:57:28.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_2 2024-04-24T23:57:28.905 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:57:29.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:29.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.211 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:29.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:29.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.212 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:29 smithi188 ceph-mon[33285]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:57:29.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:29.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:29.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:29.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:29 smithi110 ceph-mon[29187]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:29.603 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:29 smithi173 ceph-mon[32402]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:57:30.411 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:30 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi173 2024-04-24T23:57:30.411 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:30 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-24T23:57:30.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:30 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi173 2024-04-24T23:57:30.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:30 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-24T23:57:30.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:30 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi173 2024-04-24T23:57:30.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:30 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-24T23:57:30.638 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.638+0000 7f7ba3706700 1 -- 172.21.15.188:0/471143935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 msgr2=0x7f7b9c102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:30.639 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.638+0000 7f7ba3706700 1 --2- 172.21.15.188:0/471143935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c102800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009990 tx=0x7f7b8c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:57:30.639 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.638+0000 7f7ba3706700 1 -- 172.21.15.188:0/471143935 shutdown_connections 2024-04-24T23:57:30.639 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.638+0000 7f7ba3706700 1 --2- 172.21.15.188:0/471143935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c102d40 0x7f7b9c1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:30.639 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.638+0000 7f7ba3706700 1 --2- 172.21.15.188:0/471143935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9c1011a0 0x7f7b9c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:30.639 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.638+0000 7f7ba3706700 1 --2- 172.21.15.188:0/471143935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c102800 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:30.639 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.638+0000 7f7ba3706700 1 -- 172.21.15.188:0/471143935 >> 172.21.15.188:0/471143935 conn(0x7f7b9c0fc760 msgr2=0x7f7b9c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:30.639 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.639+0000 7f7ba3706700 1 -- 172.21.15.188:0/471143935 shutdown_connections 2024-04-24T23:57:30.639 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.639+0000 7f7ba3706700 1 -- 172.21.15.188:0/471143935 wait complete. 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.639+0000 7f7ba3706700 1 Processor -- start 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.639+0000 7f7ba3706700 1 -- start start 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba3706700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c1011a0 0x7f7b9c06b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba3706700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba3706700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9c102d40 0x7f7b9c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba3706700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10ccc0 con 0x7f7b9c1011a0 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba3706700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10cb40 con 0x7f7b9c102d40 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba3706700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10ce40 con 0x7f7b9c1023a0 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c19d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:33642/0 (socket says 172.21.15.188:33642) 2024-04-24T23:57:30.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 -- 172.21.15.188:0/632863758 learned_addr learned my addr 172.21.15.188:0/632863758 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba14a2700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c1011a0 0x7f7b9c06b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba1ca3700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9c102d40 0x7f7b9c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 -- 172.21.15.188:0/632863758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9c102d40 msgr2=0x7f7b9c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9c102d40 0x7f7b9c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 -- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c1011a0 msgr2=0x7f7b9c06b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c1011a0 0x7f7b9c06b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 -- 172.21.15.188:0/632863758 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c1a00c0 con 0x7f7b9c1023a0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba0ca1700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c19d1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005910 tx=0x7f7b8c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7b927fc700 1 -- 172.21.15.188:0/632863758 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c027070 con 0x7f7b9c1023a0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7b927fc700 1 -- 172.21.15.188:0/632863758 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b8c004080 con 0x7f7b9c1023a0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c1a0350 con 0x7f7b9c1023a0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7b927fc700 1 -- 172.21.15.188:0/632863758 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c022d40 con 0x7f7b9c1023a0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.640+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1a0820 con 0x7f7b9c1023a0 2024-04-24T23:57:30.643 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.641+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c02db30 con 0x7f7b9c1023a0 2024-04-24T23:57:30.644 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.643+0000 7f7b927fc700 1 -- 172.21.15.188:0/632863758 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7b8c02a070 con 0x7f7b9c1023a0 2024-04-24T23:57:30.644 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.643+0000 7f7b927fc700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7b880a2c20 0x7f7b880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:30.645 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.643+0000 7f7b927fc700 1 -- 172.21.15.188:0/632863758 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f7b8c0cf7a0 con 0x7f7b9c1023a0 2024-04-24T23:57:30.645 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.643+0000 7f7ba14a2700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7b880a2c20 0x7f7b880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:30.645 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.644+0000 7f7ba14a2700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7b880a2c20 0x7f7b880a50e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c102200 tx=0x7f7b98009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:30.646 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.647+0000 7f7b927fc700 1 -- 172.21.15.188:0/632863758 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7b8c031070 con 0x7f7b9c1023a0 2024-04-24T23:57:30.854 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:30.854+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7b9c062670 con 0x7f7b880a2c20 2024-04-24T23:57:31.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:31 smithi188 ceph-mon[33285]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:57:31.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:31 smithi188 ceph-mon[33285]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:31.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:31 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:31.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:31 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:31.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:31 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:31.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:31 smithi110 ceph-mon[29187]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:57:31.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:31 smithi110 ceph-mon[29187]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:31.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:31 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:31.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:31 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:31.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:31 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:31.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:31 smithi173 ceph-mon[32402]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:57:31.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:31 smithi173 ceph-mon[32402]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:31.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:31 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:31.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:31 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:31.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:31 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:32.473 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:32 smithi188 ceph-mon[33285]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:32.473 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:32 smithi188 ceph-mon[33285]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:57:32.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:32 smithi110 ceph-mon[29187]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:32.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:32 smithi110 ceph-mon[29187]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:57:32.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:32 smithi173 ceph-mon[32402]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:32.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:32 smithi173 ceph-mon[32402]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:57:33.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:33 smithi188 ceph-mon[33285]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:33.535 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:33 smithi110 ceph-mon[29187]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:33.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:33 smithi173 ceph-mon[32402]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:34 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/1233742279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]: dispatch 2024-04-24T23:57:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:34 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]: dispatch 2024-04-24T23:57:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:34 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]': finished 2024-04-24T23:57:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:34 smithi188 ceph-mon[33285]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T23:57:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:34.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:34 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/1233742279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]: dispatch 2024-04-24T23:57:34.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:34 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]: dispatch 2024-04-24T23:57:34.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:34 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]': finished 2024-04-24T23:57:34.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:34 smithi110 ceph-mon[29187]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T23:57:34.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:34.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:34 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/1233742279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]: dispatch 2024-04-24T23:57:34.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:34 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]: dispatch 2024-04-24T23:57:34.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:34 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2"}]': finished 2024-04-24T23:57:34.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:34 smithi173 ceph-mon[32402]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T23:57:34.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:35.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:35 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/1511189980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:57:35.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:35 smithi188 ceph-mon[33285]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:35.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:35 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/1511189980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:57:35.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:35 smithi110 ceph-mon[29187]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:35.601 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:35 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/1511189980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:57:35.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:35 smithi173 ceph-mon[32402]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:37.692 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:37 smithi188 ceph-mon[33285]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:37.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:37 smithi110 ceph-mon[29187]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:37.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:37 smithi173 ceph-mon[32402]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:39.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:39 smithi110 ceph-mon[29187]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:39.813 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:39 smithi188 ceph-mon[33285]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:39.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:39 smithi173 ceph-mon[32402]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:41 smithi110 ceph-mon[29187]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:57:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:41.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:41 smithi188 ceph-mon[33285]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:41.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:57:41.808 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:41 smithi173 ceph-mon[32402]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:57:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:42.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:42 smithi173 ceph-mon[32402]: Deploying daemon osd.8 on smithi188 2024-04-24T23:57:42.972 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:42 smithi188 ceph-mon[33285]: Deploying daemon osd.8 on smithi188 2024-04-24T23:57:43.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:42 smithi110 ceph-mon[29187]: Deploying daemon osd.8 on smithi188 2024-04-24T23:57:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:43 smithi188 ceph-mon[33285]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:43.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:43 smithi173 ceph-mon[32402]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:43 smithi110 ceph-mon[29187]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:45.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:45 smithi173 ceph-mon[32402]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:45.897 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:45 smithi188 ceph-mon[33285]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:46.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:45 smithi110 ceph-mon[29187]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:47 smithi188 ceph-mon[33285]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:47.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:47 smithi110 ceph-mon[29187]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:47.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:47.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:47.838 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:47.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:47 smithi173 ceph-mon[32402]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:49.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:49 smithi188 ceph-mon[33285]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:49 smithi173 ceph-mon[32402]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:49 smithi110 ceph-mon[29187]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:50.579 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.578+0000 7f7b927fc700 1 -- 172.21.15.188:0/632863758 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7b9c062670 con 0x7f7b880a2c20 2024-04-24T23:57:50.579 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 8 on host 'smithi188' 2024-04-24T23:57:50.581 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7b880a2c20 msgr2=0x7f7b880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7b880a2c20 0x7f7b880a50e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c102200 tx=0x7f7b98009190 comp rx=0 tx=0).stop 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 msgr2=0x7f7b9c19d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c19d1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005910 tx=0x7f7b8c009580 comp rx=0 tx=0).stop 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 shutdown_connections 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7b880a2c20 0x7f7b880a50e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9c102d40 0x7f7b9c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b9c1011a0 0x7f7b9c06b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.581+0000 7f7ba3706700 1 --2- 172.21.15.188:0/632863758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9c1023a0 0x7f7b9c19d1a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.582+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 >> 172.21.15.188:0/632863758 conn(0x7f7b9c0fc760 msgr2=0x7f7b9c105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.582+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 shutdown_connections 2024-04-24T23:57:50.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:57:50.582+0000 7f7ba3706700 1 -- 172.21.15.188:0/632863758 wait complete. 2024-04-24T23:57:51.293 DEBUG:teuthology.orchestra.run.smithi188:osd.8> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T23:57:51.297 INFO:tasks.cephadm:Deploying osd.9 on smithi110 with /dev/vg_nvme/lv_1... 2024-04-24T23:57:51.298 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:57:51.546 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:57:51.668 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:51 smithi110 ceph-mon[29187]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:51.668 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:51.668 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:51.668 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:51 smithi110 ceph-mon[29187]: from='osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:57:51.668 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:51 smithi110 ceph-mon[29187]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:57:51.668 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:51.669 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:51 smithi173 ceph-mon[32402]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:51 smithi173 ceph-mon[32402]: from='osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:57:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:51 smithi173 ceph-mon[32402]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:57:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:51 smithi188 ceph-mon[33285]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:51 smithi188 ceph-mon[33285]: from='osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:57:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:51 smithi188 ceph-mon[33285]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:57:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:52 smithi188 ceph-mon[33285]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:57:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:52 smithi188 ceph-mon[33285]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:57:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:52 smithi188 ceph-mon[33285]: from='osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:57:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:52 smithi188 ceph-mon[33285]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:57:52.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:52 smithi173 ceph-mon[32402]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:57:52.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:52 smithi173 ceph-mon[32402]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:57:52.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:52.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:52 smithi173 ceph-mon[32402]: from='osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:57:52.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:52 smithi173 ceph-mon[32402]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:57:52.868 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:52 smithi110 ceph-mon[29187]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:57:52.868 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:52 smithi110 ceph-mon[29187]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:57:52.868 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:52.868 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:52 smithi110 ceph-mon[29187]: from='osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:57:52.868 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:52 smithi110 ceph-mon[29187]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:57:53.267 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 24 23:57:52 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-24T23:57:52.956+0000 7f5a850a2700 -1 osd.8 0 waiting for initial osdmap 2024-04-24T23:57:53.267 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 24 23:57:52 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-24T23:57:52.968+0000 7f5a7ce8d700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:57:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:57:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:57:53.780 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-04-24T23:57:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-04-24T23:57:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00851286 s, 1.2 GB/s 2024-04-24T23:57:53.780 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-04-24T23:57:53.804 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:53.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:57:53.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:57:53.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:53.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:53.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: from='osd.8 ' entity='osd.8' 2024-04-24T23:57:53.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] boot 2024-04-24T23:57:53.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T23:57:53.805 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:53.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:57:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:57:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: from='osd.8 ' entity='osd.8' 2024-04-24T23:57:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] boot 2024-04-24T23:57:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T23:57:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:57:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:57:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:57:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: from='osd.8 ' entity='osd.8' 2024-04-24T23:57:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: osd.8 [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] boot 2024-04-24T23:57:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T23:57:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:57:54.506 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_1 2024-04-24T23:57:54.748 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:57:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:54 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:57:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:54 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:57:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:54 smithi110 ceph-mon[29187]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:57:54.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:54 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:57:54.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:54 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:57:54.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:54 smithi173 ceph-mon[32402]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:57:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:54 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:57:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:54 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:57:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:54 smithi188 ceph-mon[33285]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:55.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:55.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:55.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:55.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:55 smithi173 ceph-mon[32402]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:57:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:57:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:55 smithi188 ceph-mon[33285]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:56.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:57:56.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:56.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:57:56.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:57:56.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:55 smithi110 ceph-mon[29187]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:57:56.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 -- 172.21.15.110:0/137737025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b04103930 msgr2=0x7f5b04103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 --2- 172.21.15.110:0/137737025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b04103930 0x7f5b04103d50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009990 tx=0x7f5af8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 -- 172.21.15.110:0/137737025 shutdown_connections 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 --2- 172.21.15.110:0/137737025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b04100d50 0x7f5b0410a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 --2- 172.21.15.110:0/137737025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b04103930 0x7f5b04103d50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 --2- 172.21.15.110:0/137737025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b041003b0 0x7f5b04100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 -- 172.21.15.110:0/137737025 >> 172.21.15.110:0/137737025 conn(0x7f5b040fbf70 msgr2=0x7f5b040fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 -- 172.21.15.110:0/137737025 shutdown_connections 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.685+0000 7f5b0accd700 1 -- 172.21.15.110:0/137737025 wait complete. 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b0accd700 1 Processor -- start 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b0accd700 1 -- start start 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b0accd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b041003b0 0x7f5b04198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b0accd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b04100d50 0x7f5b04198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b0accd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04103930 0x7f5b0419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b0accd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0410de70 con 0x7f5b041003b0 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b0accd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0410dcf0 con 0x7f5b04103930 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b0accd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0410dff0 con 0x7f5b04100d50 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b08a69700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b041003b0 0x7f5b04198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b08a69700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b041003b0 0x7f5b04198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58800/0 (socket says 172.21.15.110:58800) 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.686+0000 7f5b08a69700 1 -- 172.21.15.110:0/709616989 learned_addr learned my addr 172.21.15.110:0/709616989 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b03fff700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b04100d50 0x7f5b04198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b08a69700 1 -- 172.21.15.110:0/709616989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b04100d50 msgr2=0x7f5b04198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b08a69700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b04100d50 0x7f5b04198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b08a69700 1 -- 172.21.15.110:0/709616989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04103930 msgr2=0x7f5b0419fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b0926a700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04103930 0x7f5b0419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b08a69700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04103930 0x7f5b0419fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b08a69700 1 -- 172.21.15.110:0/709616989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b041a0170 con 0x7f5b041003b0 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b03fff700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b04100d50 0x7f5b04198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b0926a700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04103930 0x7f5b0419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:57:56.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b08a69700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b041003b0 0x7f5b04198510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5af800bff0 tx=0x7f5af80038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b01ffb700 1 -- 172.21.15.110:0/709616989 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8027070 con 0x7f5b041003b0 2024-04-24T23:57:56.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b041a0400 con 0x7f5b041003b0 2024-04-24T23:57:56.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b01ffb700 1 -- 172.21.15.110:0/709616989 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5af8022470 con 0x7f5b041003b0 2024-04-24T23:57:56.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b01ffb700 1 -- 172.21.15.110:0/709616989 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af800f6c0 con 0x7f5b041003b0 2024-04-24T23:57:56.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.687+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b041a0850 con 0x7f5b041003b0 2024-04-24T23:57:56.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.688+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b04068f20 con 0x7f5b041003b0 2024-04-24T23:57:56.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.689+0000 7f5b01ffb700 1 -- 172.21.15.110:0/709616989 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5af802a070 con 0x7f5b041003b0 2024-04-24T23:57:56.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.690+0000 7f5b01ffb700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5af40a2c20 0x7f5af40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:57:56.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.690+0000 7f5b03fff700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5af40a2c20 0x7f5af40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:57:56.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.690+0000 7f5b01ffb700 1 -- 172.21.15.110:0/709616989 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7f5af80cfe30 con 0x7f5b041003b0 2024-04-24T23:57:56.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.690+0000 7f5b03fff700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5af40a2c20 0x7f5af40a50e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5af0007900 tx=0x7f5af0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:57:56.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.693+0000 7f5b01ffb700 1 -- 172.21.15.110:0/709616989 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5af8031070 con 0x7f5b041003b0 2024-04-24T23:57:56.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:56 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi188 2024-04-24T23:57:56.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:56 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi188 to 5696M 2024-04-24T23:57:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:56 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi188 2024-04-24T23:57:56.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:56 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi188 to 5696M 2024-04-24T23:57:56.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:57:56.900+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5b04062670 con 0x7f5af40a2c20 2024-04-24T23:57:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:56 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi188 2024-04-24T23:57:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:56 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi188 to 5696M 2024-04-24T23:57:57.767 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:57 smithi110 ceph-mon[29187]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:57:57.767 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:57 smithi110 ceph-mon[29187]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:57:57.767 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:57.767 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:57.767 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:57 smithi173 ceph-mon[32402]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:57:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:57 smithi173 ceph-mon[32402]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:57:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:57.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:57 smithi188 ceph-mon[33285]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:57:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:57 smithi188 ceph-mon[33285]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:57:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:57:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:57:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:57:58.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:58 smithi110 ceph-mon[29187]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:58.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:58 smithi110 ceph-mon[29187]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:57:58.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:58 smithi173 ceph-mon[32402]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:58.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:58 smithi173 ceph-mon[32402]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:57:59.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:58 smithi188 ceph-mon[33285]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:57:59.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:58 smithi188 ceph-mon[33285]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:57:59.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:57:59 smithi173 ceph-mon[32402]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-24T23:58:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:57:59 smithi188 ceph-mon[33285]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-24T23:58:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:57:59 smithi110 ceph-mon[29187]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-24T23:58:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:00 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3612499683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d5cf0eb-8bf8-488a-8183-f8c9961006f7"}]: dispatch 2024-04-24T23:58:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:00 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3612499683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d5cf0eb-8bf8-488a-8183-f8c9961006f7"}]': finished 2024-04-24T23:58:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:00 smithi173 ceph-mon[32402]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T23:58:00.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:00.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:00 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3080928518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:01.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:00 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3612499683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d5cf0eb-8bf8-488a-8183-f8c9961006f7"}]: dispatch 2024-04-24T23:58:01.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:00 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3612499683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d5cf0eb-8bf8-488a-8183-f8c9961006f7"}]': finished 2024-04-24T23:58:01.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:00 smithi188 ceph-mon[33285]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T23:58:01.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:01.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:00 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3080928518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:00 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3612499683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d5cf0eb-8bf8-488a-8183-f8c9961006f7"}]: dispatch 2024-04-24T23:58:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:00 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3612499683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d5cf0eb-8bf8-488a-8183-f8c9961006f7"}]': finished 2024-04-24T23:58:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:00 smithi110 ceph-mon[29187]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T23:58:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:00 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3080928518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:01.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:01 smithi173 ceph-mon[32402]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-24T23:58:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:01 smithi188 ceph-mon[33285]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-24T23:58:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:01 smithi110 ceph-mon[29187]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-24T23:58:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:03 smithi110 ceph-mon[29187]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:58:03.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:03 smithi173 ceph-mon[32402]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:58:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:03 smithi188 ceph-mon[33285]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:58:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:05 smithi173 ceph-mon[32402]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-24T23:58:05.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:05 smithi110 ceph-mon[29187]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-24T23:58:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:05 smithi188 ceph-mon[33285]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-24T23:58:07.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:07 smithi173 ceph-mon[32402]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:07 smithi188 ceph-mon[33285]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:07 smithi110 ceph-mon[29187]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:08.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:58:08.855 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:08 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:58:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:08 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:58:09.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:08 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:09.862 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:09 smithi110 ceph-mon[29187]: Deploying daemon osd.9 on smithi110 2024-04-24T23:58:09.862 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:09 smithi110 ceph-mon[29187]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:09 smithi188 ceph-mon[33285]: Deploying daemon osd.9 on smithi110 2024-04-24T23:58:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:09 smithi188 ceph-mon[33285]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:09 smithi173 ceph-mon[32402]: Deploying daemon osd.9 on smithi110 2024-04-24T23:58:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:09 smithi173 ceph-mon[32402]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:11 smithi110 ceph-mon[29187]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:11 smithi188 ceph-mon[33285]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:11 smithi173 ceph-mon[32402]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:13 smithi188 ceph-mon[33285]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:13 smithi110 ceph-mon[29187]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:14.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:13 smithi173 ceph-mon[32402]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:14.811 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:14.811 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:14.811 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:15.842 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:15 smithi110 ceph-mon[29187]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:15 smithi188 ceph-mon[33285]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:16.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:15 smithi173 ceph-mon[32402]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:17.726 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 9 on host 'smithi110' 2024-04-24T23:58:17.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.726+0000 7f5b01ffb700 1 -- 172.21.15.110:0/709616989 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b04062670 con 0x7f5af40a2c20 2024-04-24T23:58:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.728+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5af40a2c20 msgr2=0x7f5af40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:17.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.728+0000 7f5b0accd700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5af40a2c20 0x7f5af40a50e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5af0007900 tx=0x7f5af0008040 comp rx=0 tx=0).stop 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.728+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b041003b0 msgr2=0x7f5b04198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.728+0000 7f5b0accd700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b041003b0 0x7f5b04198510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5af800bff0 tx=0x7f5af80038c0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.728+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 shutdown_connections 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.728+0000 7f5b0accd700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5af40a2c20 0x7f5af40a50e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.728+0000 7f5b0accd700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04103930 0x7f5b0419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.728+0000 7f5b0accd700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b041003b0 0x7f5b04198510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.729+0000 7f5b0accd700 1 --2- 172.21.15.110:0/709616989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b04100d50 0x7f5b04198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.729+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 >> 172.21.15.110:0/709616989 conn(0x7f5b040fbf70 msgr2=0x7f5b04106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.729+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 shutdown_connections 2024-04-24T23:58:17.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:58:17.729+0000 7f5b0accd700 1 -- 172.21.15.110:0/709616989 wait complete. 2024-04-24T23:58:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:17 smithi188 ceph-mon[33285]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:17 smithi110 ceph-mon[29187]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:17 smithi173 ceph-mon[32402]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:18.631 DEBUG:teuthology.orchestra.run.smithi110:osd.9> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T23:58:18.633 INFO:tasks.cephadm:Deploying osd.10 on smithi173 with /dev/vg_nvme/lv_1... 2024-04-24T23:58:18.633 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:58:18.882 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:58:18.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:18 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:18.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:18 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:18.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:18 smithi110 ceph-mon[29187]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:58:18.926 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:18 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:18.926 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:18 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:18.926 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:18 smithi173 ceph-mon[32402]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:58:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:18 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:18 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:18 smithi188 ceph-mon[33285]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:58:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:19 smithi173 ceph-mon[32402]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:19 smithi173 ceph-mon[32402]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:58:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:19 smithi173 ceph-mon[32402]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:58:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:19 smithi173 ceph-mon[32402]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:58:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:19 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:19 smithi188 ceph-mon[33285]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:19 smithi188 ceph-mon[33285]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:58:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:19 smithi188 ceph-mon[33285]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:58:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:19 smithi188 ceph-mon[33285]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:58:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:20.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:19 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-mon[29187]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-mon[29187]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:58:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-mon[29187]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:58:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-mon[29187]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-24T23:58:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:20.036 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-24T23:58:19.692+0000 7f6f1000a700 -1 osd.9 0 waiting for initial osdmap 2024-04-24T23:58:20.037 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 24 23:58:19 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-24T23:58:19.704+0000 7f6f0bdfd700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:20.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:20 smithi173 ceph-mon[32402]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:58:20.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:20 smithi173 ceph-mon[32402]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:58:20.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:20 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:20.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:20 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:20.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:20 smithi173 ceph-mon[32402]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:21.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:20 smithi188 ceph-mon[33285]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:58:21.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:20 smithi188 ceph-mon[33285]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:58:21.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:20 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:21.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:20 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:21.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:20 smithi188 ceph-mon[33285]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:21.023 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:58:21.023 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:58:21.023 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-24T23:58:21.023 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-24T23:58:21.023 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112587 s, 931 MB/s 2024-04-24T23:58:21.023 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-24T23:58:21.024 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:20 smithi110 ceph-mon[29187]: from='osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-24T23:58:21.024 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:20 smithi110 ceph-mon[29187]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:58:21.024 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:20 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:21.024 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:20 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:21.024 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:20 smithi110 ceph-mon[29187]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:58:21.775 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_1 2024-04-24T23:58:22.016 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:58:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:21 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:58:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:21 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:58:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:21 smithi188 ceph-mon[33285]: osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] boot 2024-04-24T23:58:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:21 smithi188 ceph-mon[33285]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T23:58:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:21 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:21 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:58:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:21 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:58:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:21 smithi110 ceph-mon[29187]: osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] boot 2024-04-24T23:58:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:21 smithi110 ceph-mon[29187]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T23:58:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:21 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:21 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:58:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:21 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:58:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:21 smithi173 ceph-mon[32402]: osd.9 [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] boot 2024-04-24T23:58:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:21 smithi173 ceph-mon[32402]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T23:58:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:21 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:58:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:58:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi110 2024-04-24T23:58:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:22.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:22.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:22.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi110 to 4272M 2024-04-24T23:58:22.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:22.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:22.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:22.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:22 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:58:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi110 2024-04-24T23:58:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi110 to 4272M 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:23.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:22 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi110 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi110 to 4272M 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:23.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:23.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:22 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 -- 172.21.15.173:0/2048161231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102390 msgr2=0x7f0acc1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2048161231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102390 0x7f0acc1027f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40099c0 tx=0x7f0ab4009cd0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 -- 172.21.15.173:0/2048161231 shutdown_connections 2024-04-24T23:58:23.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2048161231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc102d30 0x7f0acc1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2048161231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc101190 0x7f0acc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2048161231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102390 0x7f0acc1027f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 -- 172.21.15.173:0/2048161231 >> 172.21.15.173:0/2048161231 conn(0x7f0acc0fc730 msgr2=0x7f0acc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 -- 172.21.15.173:0/2048161231 shutdown_connections 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 -- 172.21.15.173:0/2048161231 wait complete. 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.714+0000 7f0ad1383700 1 Processor -- start 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ad1383700 1 -- start start 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ad1383700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc101190 0x7f0acc06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ad1383700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc102390 0x7f0acc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ad1383700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102d30 0x7f0acc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ad1383700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10ccc0 con 0x7f0acc102390 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ad1383700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10cb40 con 0x7f0acc102d30 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ad1383700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10ce40 con 0x7f0acc101190 2024-04-24T23:58:23.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102d30 0x7f0acc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102d30 0x7f0acc19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:39450/0 (socket says 172.21.15.173:39450) 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 -- 172.21.15.173:0/2745893646 learned_addr learned my addr 172.21.15.173:0/2745893646 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0aca7fc700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc102390 0x7f0acc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acaffd700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc101190 0x7f0acc06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 -- 172.21.15.173:0/2745893646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc101190 msgr2=0x7f0acc06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc101190 0x7f0acc06aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 -- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc102390 msgr2=0x7f0acc19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc102390 0x7f0acc19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 -- 172.21.15.173:0/2745893646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc1a0120 con 0x7f0acc102d30 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0aca7fc700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc102390 0x7f0acc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:58:23.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0acb7fe700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102d30 0x7f0acc19f850 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400d980 tx=0x7f0ac400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.719 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ac3fff700 1 -- 172.21.15.173:0/2745893646 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4017070 con 0x7f0acc102d30 2024-04-24T23:58:23.719 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ac3fff700 1 -- 172.21.15.173:0/2745893646 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ac4009df0 con 0x7f0acc102d30 2024-04-24T23:58:23.719 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.715+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc1a0410 con 0x7f0acc102d30 2024-04-24T23:58:23.719 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.716+0000 7f0ac3fff700 1 -- 172.21.15.173:0/2745893646 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40053a0 con 0x7f0acc102d30 2024-04-24T23:58:23.719 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.716+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc1a09d0 con 0x7f0acc102d30 2024-04-24T23:58:23.719 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.716+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0acc068f20 con 0x7f0acc102d30 2024-04-24T23:58:23.720 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.718+0000 7f0ac3fff700 1 -- 172.21.15.173:0/2745893646 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0ac4004750 con 0x7f0acc102d30 2024-04-24T23:58:23.720 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.718+0000 7f0ac3fff700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0ab80a2c20 0x7f0ab80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:23.720 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.718+0000 7f0ac3fff700 1 -- 172.21.15.173:0/2745893646 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f0ac40cecb0 con 0x7f0acc102d30 2024-04-24T23:58:23.720 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.718+0000 7f0acaffd700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0ab80a2c20 0x7f0ab80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:23.720 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.719+0000 7f0acaffd700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0ab80a2c20 0x7f0ab80a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0acc1021f0 tx=0x7f0abc0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:23.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.722+0000 7f0ac3fff700 1 -- 172.21.15.173:0/2745893646 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0ac4020020 con 0x7f0acc102d30 2024-04-24T23:58:23.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:23.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:23.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:23.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:23.928 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:23.928+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0acc062670 con 0x7f0ab80a2c20 2024-04-24T23:58:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:24.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:24 smithi173 ceph-mon[32402]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:58:24.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:24 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:58:24.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:24 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:58:24.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:24 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:24.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:24 smithi173 ceph-mon[32402]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:24 smithi188 ceph-mon[33285]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:58:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:24 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:58:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:24 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:58:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:24 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:24 smithi188 ceph-mon[33285]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:25.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:24 smithi110 ceph-mon[29187]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:58:25.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:24 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:58:25.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:24 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:58:25.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:24 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:25.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:24 smithi110 ceph-mon[29187]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:27 smithi173 ceph-mon[32402]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:27 smithi173 ceph-mon[32402]: from='client.? 172.21.15.173:0/2165028034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]: dispatch 2024-04-24T23:58:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:27 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]: dispatch 2024-04-24T23:58:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:27 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]': finished 2024-04-24T23:58:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:27 smithi173 ceph-mon[32402]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T23:58:27.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:27.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:27 smithi173 ceph-mon[32402]: from='client.? 172.21.15.173:0/2690592724' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:27 smithi188 ceph-mon[33285]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:27 smithi188 ceph-mon[33285]: from='client.? 172.21.15.173:0/2165028034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]: dispatch 2024-04-24T23:58:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:27 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]: dispatch 2024-04-24T23:58:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:27 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]': finished 2024-04-24T23:58:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:27 smithi188 ceph-mon[33285]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T23:58:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:27 smithi188 ceph-mon[33285]: from='client.? 172.21.15.173:0/2690592724' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:27 smithi110 ceph-mon[29187]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:27 smithi110 ceph-mon[29187]: from='client.? 172.21.15.173:0/2165028034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]: dispatch 2024-04-24T23:58:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:27 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]: dispatch 2024-04-24T23:58:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:27 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51"}]': finished 2024-04-24T23:58:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:27 smithi110 ceph-mon[29187]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T23:58:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:27 smithi110 ceph-mon[29187]: from='client.? 172.21.15.173:0/2690592724' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:29.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:29 smithi110 ceph-mon[29187]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:29.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:29 smithi173 ceph-mon[32402]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:29 smithi188 ceph-mon[33285]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:31.796 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:31 smithi173 ceph-mon[32402]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:31 smithi188 ceph-mon[33285]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:31 smithi110 ceph-mon[29187]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:33.747 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:33 smithi173 ceph-mon[32402]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:33 smithi188 ceph-mon[33285]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:34.035 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:33 smithi110 ceph-mon[29187]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:34.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:58:34.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:58:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:58:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:35.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:35 smithi173 ceph-mon[32402]: Deploying daemon osd.10 on smithi173 2024-04-24T23:58:35.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:35 smithi173 ceph-mon[32402]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:35.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:35 smithi188 ceph-mon[33285]: Deploying daemon osd.10 on smithi173 2024-04-24T23:58:35.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:35 smithi188 ceph-mon[33285]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:35 smithi110 ceph-mon[29187]: Deploying daemon osd.10 on smithi173 2024-04-24T23:58:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:35 smithi110 ceph-mon[29187]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:37.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:37 smithi173 ceph-mon[32402]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:37 smithi188 ceph-mon[33285]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:37 smithi110 ceph-mon[29187]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:39.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:39 smithi173 ceph-mon[32402]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:39 smithi188 ceph-mon[33285]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:39 smithi110 ceph-mon[29187]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:40 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:40 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:40 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:40 smithi173 ceph-mon[32402]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:40 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:40 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:41.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:40 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:41.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:40 smithi188 ceph-mon[33285]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:41.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:40 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:41.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:40 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:41.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:40 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:41.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:40 smithi110 ceph-mon[29187]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:43.695 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:43 smithi173 ceph-mon[32402]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:43.695 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:43.695 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:43.695 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:43.695 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:43.695 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:43 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:43.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.744+0000 7f0ac3fff700 1 -- 172.21.15.173:0/2745893646 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0acc062670 con 0x7f0ab80a2c20 2024-04-24T23:58:43.744 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 10 on host 'smithi173' 2024-04-24T23:58:43.747 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0ab80a2c20 msgr2=0x7f0ab80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.747 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0ab80a2c20 0x7f0ab80a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0acc1021f0 tx=0x7f0abc0094f0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.747 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102d30 msgr2=0x7f0acc19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:43.747 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102d30 0x7f0acc19f850 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400d980 tx=0x7f0ac400dd40 comp rx=0 tx=0).stop 2024-04-24T23:58:43.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 shutdown_connections 2024-04-24T23:58:43.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0ab80a2c20 0x7f0ab80a50e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0acc102d30 0x7f0acc19f850 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0acc101190 0x7f0acc06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 --2- 172.21.15.173:0/2745893646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc102390 0x7f0acc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:43.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 >> 172.21.15.173:0/2745893646 conn(0x7f0acc0fc730 msgr2=0x7f0acc1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:43.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 shutdown_connections 2024-04-24T23:58:43.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-24T23:58:43.747+0000 7f0ad1383700 1 -- 172.21.15.173:0/2745893646 wait complete. 2024-04-24T23:58:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:43 smithi188 ceph-mon[33285]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:43 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:43 smithi110 ceph-mon[29187]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:44.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:44.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:43 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.425 DEBUG:teuthology.orchestra.run.smithi173:osd.10> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T23:58:44.427 INFO:tasks.cephadm:Deploying osd.11 on smithi188 with /dev/vg_nvme/lv_1... 2024-04-24T23:58:44.427 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:58:44.674 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:58:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:44 smithi173 ceph-mon[32402]: from='osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:58:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:44 smithi173 ceph-mon[32402]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:58:44.724 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.725 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:44.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:44 smithi188 ceph-mon[33285]: from='osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:58:44.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:44 smithi188 ceph-mon[33285]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:58:44.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:44.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:44 smithi110 ceph-mon[29187]: from='osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:58:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:44 smithi110 ceph-mon[29187]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:58:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:58:45.811 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:45 smithi188 ceph-mon[33285]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:45.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:45 smithi188 ceph-mon[33285]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:58:45.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:45 smithi188 ceph-mon[33285]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:58:45.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:45 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:45.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:45 smithi188 ceph-mon[33285]: from='osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:58:45.812 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:45 smithi188 ceph-mon[33285]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:58:45.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:45 smithi173 ceph-mon[32402]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:45.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:45 smithi173 ceph-mon[32402]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:58:45.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:45 smithi173 ceph-mon[32402]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:58:45.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:45 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:45.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:45 smithi173 ceph-mon[32402]: from='osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:58:45.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:45 smithi173 ceph-mon[32402]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:58:45.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 24 23:58:45 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-24T23:58:45.566+0000 7ff653654700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T23:58:45.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 24 23:58:45 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-24T23:58:45.583+0000 7ff64f447700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:58:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:45 smithi110 ceph-mon[29187]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:45 smithi110 ceph-mon[29187]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:58:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:45 smithi110 ceph-mon[29187]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:58:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:45 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:45 smithi110 ceph-mon[29187]: from='osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:58:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:45 smithi110 ceph-mon[29187]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-24T23:58:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:46 smithi188 ceph-mon[33285]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:58:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:46 smithi188 ceph-mon[33285]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:58:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:46 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:46 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:46.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:46 smithi173 ceph-mon[32402]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:58:46.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:46 smithi173 ceph-mon[32402]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:58:46.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:46 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:46.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:46 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:46.891 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:58:46.892 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:58:46.892 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-04-24T23:58:46.892 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-04-24T23:58:46.892 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110914 s, 945 MB/s 2024-04-24T23:58:46.892 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-04-24T23:58:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:46 smithi110 ceph-mon[29187]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-24T23:58:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:46 smithi110 ceph-mon[29187]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:58:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:46 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:47.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:46 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:47.605 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_1 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] boot 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:47.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:47.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.853 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:58:47.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:58:47.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] boot 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:47.888 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: osd.10 [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] boot 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.938 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.939 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.939 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:58:47.939 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:47.939 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:47.939 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:58:47.939 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:58:48.844 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:48 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi173 2024-04-24T23:58:48.844 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:48 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-24T23:58:48.844 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:48 smithi188 ceph-mon[33285]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:58:48.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:48 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi173 2024-04-24T23:58:48.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:48 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-24T23:58:48.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:48 smithi173 ceph-mon[32402]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:58:48.957 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:48 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi173 2024-04-24T23:58:48.957 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:48 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-24T23:58:48.957 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:48 smithi110 ceph-mon[29187]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:58:49.574 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 -- 172.21.15.188:0/3989488107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102380 msgr2=0x7f92641027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.574 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 --2- 172.21.15.188:0/3989488107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102380 0x7f92641027e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9254009990 tx=0x7f9254009ca0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 -- 172.21.15.188:0/3989488107 shutdown_connections 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 --2- 172.21.15.188:0/3989488107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9264102d20 0x7f92641095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 --2- 172.21.15.188:0/3989488107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9264101180 0x7f92641015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 --2- 172.21.15.188:0/3989488107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102380 0x7f92641027e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 -- 172.21.15.188:0/3989488107 >> 172.21.15.188:0/3989488107 conn(0x7f92640fc760 msgr2=0x7f92640feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 -- 172.21.15.188:0/3989488107 shutdown_connections 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 -- 172.21.15.188:0/3989488107 wait complete. 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 Processor -- start 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.573+0000 7f926bde1700 1 -- start start 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926bde1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9264101180 0x7f926419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926bde1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9264102380 0x7f926419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926bde1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102d20 0x7f92641a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.575 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926bde1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f926410cc20 con 0x7f9264102380 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926bde1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f926410caa0 con 0x7f9264101180 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926bde1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f926410cda0 con 0x7f9264102d20 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102d20 0x7f92641a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926937c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9264102380 0x7f926419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102d20 0x7f92641a1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:54512/0 (socket says 172.21.15.188:54512) 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 -- 172.21.15.188:0/2515411271 learned_addr learned my addr 172.21.15.188:0/2515411271 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f9269b7d700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9264101180 0x7f926419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 -- 172.21.15.188:0/2515411271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9264101180 msgr2=0x7f926419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9264101180 0x7f926419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 -- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9264102380 msgr2=0x7f926419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9264102380 0x7f926419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 -- 172.21.15.188:0/2515411271 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92641a2260 con 0x7f9264102d20 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926a37e700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102d20 0x7f92641a1bc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f925c007f10 tx=0x7f925c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:49.576 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f925affd700 1 -- 172.21.15.188:0/2515411271 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c004bb0 con 0x7f9264102d20 2024-04-24T23:58:49.577 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f925affd700 1 -- 172.21.15.188:0/2515411271 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f925c004d50 con 0x7f9264102d20 2024-04-24T23:58:49.577 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92641a2550 con 0x7f9264102d20 2024-04-24T23:58:49.578 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f925affd700 1 -- 172.21.15.188:0/2515411271 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c0058e0 con 0x7f9264102d20 2024-04-24T23:58:49.578 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.574+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92641a2a40 con 0x7f9264102d20 2024-04-24T23:58:49.578 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.575+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9264068f20 con 0x7f9264102d20 2024-04-24T23:58:49.578 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.577+0000 7f925affd700 1 -- 172.21.15.188:0/2515411271 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f925c01a020 con 0x7f9264102d20 2024-04-24T23:58:49.578 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.577+0000 7f925affd700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f92500a2c20 0x7f92500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:58:49.578 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.577+0000 7f9269b7d700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f92500a2c20 0x7f92500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:58:49.578 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.577+0000 7f925affd700 1 -- 172.21.15.188:0/2515411271 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f925c0ce740 con 0x7f9264102d20 2024-04-24T23:58:49.578 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.578+0000 7f9269b7d700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f92500a2c20 0x7f92500a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f92641021e0 tx=0x7f9260009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:58:49.580 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.581+0000 7f925affd700 1 -- 172.21.15.188:0/2515411271 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f925c010070 con 0x7f9264102d20 2024-04-24T23:58:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:49 smithi188 ceph-mon[33285]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:58:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:49 smithi188 ceph-mon[33285]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:58:49.790 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:58:49.790+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f92641a2d60 con 0x7f92500a2c20 2024-04-24T23:58:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:49 smithi173 ceph-mon[32402]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:58:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:49 smithi173 ceph-mon[32402]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:58:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:49 smithi110 ceph-mon[29187]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:58:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:49 smithi110 ceph-mon[29187]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:58:50.758 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:50 smithi188 ceph-mon[33285]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:58:50.758 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:50 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:58:50.758 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:50 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:58:50.758 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:50 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:50 smithi173 ceph-mon[32402]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:58:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:50 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:58:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:50 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:58:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:50 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:51.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:50 smithi110 ceph-mon[29187]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:58:51.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:50 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:58:51.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:50 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:58:51.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:50 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:58:51.757 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:51 smithi188 ceph-mon[33285]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:58:51.757 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:51 smithi188 ceph-mon[33285]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:58:51.757 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:51 smithi188 ceph-mon[33285]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:58:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:51 smithi173 ceph-mon[32402]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:58:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:51 smithi173 ceph-mon[32402]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:58:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:51 smithi173 ceph-mon[32402]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:58:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:51 smithi110 ceph-mon[29187]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:58:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:51 smithi110 ceph-mon[29187]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:58:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:51 smithi110 ceph-mon[29187]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:58:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:53 smithi173 ceph-mon[32402]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-24T23:58:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:53 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/1178749472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]: dispatch 2024-04-24T23:58:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:53 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]: dispatch 2024-04-24T23:58:53.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:53 smithi173 ceph-mon[32402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]': finished 2024-04-24T23:58:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:53 smithi173 ceph-mon[32402]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T23:58:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:58:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:53 smithi173 ceph-mon[32402]: from='client.? 172.21.15.188:0/923870638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:53 smithi188 ceph-mon[33285]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-24T23:58:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:53 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/1178749472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]: dispatch 2024-04-24T23:58:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:53 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]: dispatch 2024-04-24T23:58:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:53 smithi188 ceph-mon[33285]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]': finished 2024-04-24T23:58:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:53 smithi188 ceph-mon[33285]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T23:58:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:58:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:53 smithi188 ceph-mon[33285]: from='client.? 172.21.15.188:0/923870638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:53 smithi110 ceph-mon[29187]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-24T23:58:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:53 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/1178749472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]: dispatch 2024-04-24T23:58:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:53 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]: dispatch 2024-04-24T23:58:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:53 smithi110 ceph-mon[29187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f2ae40f-4939-417d-b66f-990914900d30"}]': finished 2024-04-24T23:58:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:53 smithi110 ceph-mon[29187]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T23:58:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:58:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:53 smithi110 ceph-mon[29187]: from='client.? 172.21.15.188:0/923870638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:58:55.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:55 smithi173 ceph-mon[32402]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:58:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:55 smithi188 ceph-mon[33285]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:58:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:55 smithi110 ceph-mon[29187]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:58:57.970 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:57 smithi188 ceph-mon[33285]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:58:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:57 smithi110 ceph-mon[29187]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:58:58.101 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:57 smithi173 ceph-mon[32402]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:58:59.871 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:58:59 smithi188 ceph-mon[33285]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:59:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:58:59 smithi110 ceph-mon[29187]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:59:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:58:59 smithi173 ceph-mon[32402]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:59:00.944 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:59:00.944 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:59:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:01.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:59:01.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:01.994 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:01 smithi188 ceph-mon[33285]: Deploying daemon osd.11 on smithi188 2024-04-24T23:59:01.995 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:01 smithi188 ceph-mon[33285]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:59:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:01 smithi110 ceph-mon[29187]: Deploying daemon osd.11 on smithi188 2024-04-24T23:59:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:01 smithi110 ceph-mon[29187]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:59:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:01 smithi173 ceph-mon[32402]: Deploying daemon osd.11 on smithi188 2024-04-24T23:59:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:01 smithi173 ceph-mon[32402]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:59:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:03 smithi188 ceph-mon[33285]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:03 smithi110 ceph-mon[29187]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:03 smithi173 ceph-mon[32402]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:05.975 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:05 smithi188 ceph-mon[33285]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:05 smithi110 ceph-mon[29187]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:05 smithi173 ceph-mon[32402]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:07.496 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:07.496 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:07.496 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:07.496 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:07 smithi188 ceph-mon[33285]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:07.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:07 smithi110 ceph-mon[29187]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:07.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:07.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:07.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:07.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:07 smithi173 ceph-mon[32402]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:09.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:09 smithi173 ceph-mon[32402]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:09 smithi188 ceph-mon[33285]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:09 smithi110 ceph-mon[29187]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:10.417 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 11 on host 'smithi188' 2024-04-24T23:59:10.417 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.416+0000 7f925affd700 1 -- 172.21.15.188:0/2515411271 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f92641a2d60 con 0x7f92500a2c20 2024-04-24T23:59:10.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f92500a2c20 msgr2=0x7f92500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f92500a2c20 0x7f92500a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f92641021e0 tx=0x7f9260009260 comp rx=0 tx=0).stop 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102d20 msgr2=0x7f92641a1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102d20 0x7f92641a1bc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f925c007f10 tx=0x7f925c00d3b0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 shutdown_connections 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f92500a2c20 0x7f92500a50e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9264102d20 0x7f92641a1bc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9264101180 0x7f926419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 --2- 172.21.15.188:0/2515411271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9264102380 0x7f926419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 >> 172.21.15.188:0/2515411271 conn(0x7f92640fc760 msgr2=0x7f92640fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 shutdown_connections 2024-04-24T23:59:10.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:10.419+0000 7f926bde1700 1 -- 172.21.15.188:0/2515411271 wait complete. 2024-04-24T23:59:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:59:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:59:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:10 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:11.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:11.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:11.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:59:11.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:59:11.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:10 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:11.077 DEBUG:teuthology.orchestra.run.smithi188:osd.11> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T23:59:11.080 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T23:59:11.080 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:59:11.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:11.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:11.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:59:11.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:59:11.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:11.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:10 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:11.334 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:59:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:11 smithi110 ceph-mon[29187]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:11 smithi110 ceph-mon[29187]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:59:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:11 smithi110 ceph-mon[29187]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:59:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:11 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:11 smithi110 ceph-mon[29187]: from='osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:59:11.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:11 smithi110 ceph-mon[29187]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:59:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:11 smithi188 ceph-mon[33285]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:11 smithi188 ceph-mon[33285]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:59:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:11 smithi188 ceph-mon[33285]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:59:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:11 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:11 smithi188 ceph-mon[33285]: from='osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:59:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:11 smithi188 ceph-mon[33285]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:59:12.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 24 23:59:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-24T23:59:11.654+0000 7f51e6404700 -1 osd.11 0 waiting for initial osdmap 2024-04-24T23:59:12.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 24 23:59:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-24T23:59:11.666+0000 7f51e21f7700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:59:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:11 smithi173 ceph-mon[32402]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:11 smithi173 ceph-mon[32402]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:59:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:11 smithi173 ceph-mon[32402]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:59:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:11 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:11 smithi173 ceph-mon[32402]: from='osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:59:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:11 smithi173 ceph-mon[32402]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-24T23:59:12.908 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:12 smithi110 ceph-mon[29187]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:59:12.908 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:12 smithi110 ceph-mon[29187]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:59:12.908 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:12.908 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:12 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:13.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:12 smithi188 ceph-mon[33285]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:59:13.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:12 smithi188 ceph-mon[33285]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:59:13.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:13.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:12 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:13.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:12 smithi173 ceph-mon[32402]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-24T23:59:13.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:12 smithi173 ceph-mon[32402]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:59:13.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:13.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:12 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:13.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/1811482167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 msgr2=0x7f0e38102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/1811482167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 0x7f0e38102700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0099c0 tx=0x7f0e2c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/1811482167 shutdown_connections 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/1811482167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38103e80 0x7f0e3810a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/1811482167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 0x7f0e38102700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/1811482167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e381034e0 0x7f0e38103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/1811482167 >> 172.21.15.110:0/1811482167 conn(0x7f0e380fd880 msgr2=0x7f0e380ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/1811482167 shutdown_connections 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.283+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/1811482167 wait complete. 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3eeaf700 1 Processor -- start 2024-04-24T23:59:13.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3eeaf700 1 -- start start 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3eeaf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 0x7f0e381983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3eeaf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e381034e0 0x7f0e38198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3eeaf700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38103e80 0x7f0e3819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3eeaf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3810dd70 con 0x7f0e381022e0 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3eeaf700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3810dbf0 con 0x7f0e38103e80 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3eeaf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3810def0 con 0x7f0e381034e0 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3cc4b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 0x7f0e381983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3cc4b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 0x7f0e381983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37464/0 (socket says 172.21.15.110:37464) 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3cc4b700 1 -- 172.21.15.110:0/4011751738 learned_addr learned my addr 172.21.15.110:0/4011751738 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e37fff700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e381034e0 0x7f0e38198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3d44c700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38103e80 0x7f0e3819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e37fff700 1 -- 172.21.15.110:0/4011751738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38103e80 msgr2=0x7f0e3819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e37fff700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38103e80 0x7f0e3819f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e37fff700 1 -- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 msgr2=0x7f0e381983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e37fff700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 0x7f0e381983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e37fff700 1 -- 172.21.15.110:0/4011751738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e381a0040 con 0x7f0e381034e0 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e3cc4b700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 0x7f0e381983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.284+0000 7f0e37fff700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e381034e0 0x7f0e38198920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400ea20 tx=0x7f0e2400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.285+0000 7f0e35ffb700 1 -- 172.21.15.110:0/4011751738 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400cbd0 con 0x7f0e381034e0 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.285+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e381a0330 con 0x7f0e381034e0 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.285+0000 7f0e35ffb700 1 -- 172.21.15.110:0/4011751738 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e2400ee60 con 0x7f0e381034e0 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.285+0000 7f0e35ffb700 1 -- 172.21.15.110:0/4011751738 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2401c070 con 0x7f0e381034e0 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.285+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e381a0870 con 0x7f0e381034e0 2024-04-24T23:59:13.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.286+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e38068f20 con 0x7f0e381034e0 2024-04-24T23:59:13.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.287+0000 7f0e35ffb700 1 -- 172.21.15.110:0/4011751738 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0e24043b30 con 0x7f0e381034e0 2024-04-24T23:59:13.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.287+0000 7f0e35ffb700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0e280a2c20 0x7f0e280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:13.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.287+0000 7f0e35ffb700 1 -- 172.21.15.110:0/4011751738 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f0e240ceae0 con 0x7f0e381034e0 2024-04-24T23:59:13.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.287+0000 7f0e3cc4b700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0e280a2c20 0x7f0e280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:13.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.288+0000 7f0e3cc4b700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0e280a2c20 0x7f0e280a50e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c000c00 tx=0x7f0e2c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:13.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.291+0000 7f0e35ffb700 1 -- 172.21.15.110:0/4011751738 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e2409be40 con 0x7f0e381034e0 2024-04-24T23:59:13.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.497+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0e3802db60 con 0x7f0e381034e0 2024-04-24T23:59:13.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.498+0000 7f0e35ffb700 1 -- 172.21.15.110:0/4011751738 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f0e24043e20 con 0x7f0e381034e0 2024-04-24T23:59:13.501 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0e280a2c20 msgr2=0x7f0e280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0e280a2c20 0x7f0e280a50e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c000c00 tx=0x7f0e2c005fb0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e381034e0 msgr2=0x7f0e38198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e381034e0 0x7f0e38198920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400ea20 tx=0x7f0e2400ed30 comp rx=0 tx=0).stop 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 shutdown_connections 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0e280a2c20 0x7f0e280a50e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38103e80 0x7f0e3819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381022e0 0x7f0e381983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 --2- 172.21.15.110:0/4011751738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e381034e0 0x7f0e38198920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:13.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 >> 172.21.15.110:0/4011751738 conn(0x7f0e380fd880 msgr2=0x7f0e38106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:13.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 shutdown_connections 2024-04-24T23:59:13.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:13.500+0000 7f0e3eeaf700 1 -- 172.21.15.110:0/4011751738 wait complete. 2024-04-24T23:59:13.658 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:13 smithi188 ceph-mon[33285]: purged_snaps scrub starts 2024-04-24T23:59:13.658 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:13 smithi188 ceph-mon[33285]: purged_snaps scrub ok 2024-04-24T23:59:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:13 smithi110 ceph-mon[29187]: purged_snaps scrub starts 2024-04-24T23:59:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:13 smithi110 ceph-mon[29187]: purged_snaps scrub ok 2024-04-24T23:59:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:13 smithi110 ceph-mon[29187]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:13 smithi110 ceph-mon[29187]: osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] boot 2024-04-24T23:59:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:13 smithi110 ceph-mon[29187]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T23:59:13.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:13 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:13.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:13 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4011751738' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:59:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:13 smithi188 ceph-mon[33285]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:13 smithi188 ceph-mon[33285]: osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] boot 2024-04-24T23:59:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:13 smithi188 ceph-mon[33285]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T23:59:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:13 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:13 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4011751738' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:59:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:13 smithi173 ceph-mon[32402]: purged_snaps scrub starts 2024-04-24T23:59:14.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:13 smithi173 ceph-mon[32402]: purged_snaps scrub ok 2024-04-24T23:59:14.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:13 smithi173 ceph-mon[32402]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:59:14.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:13 smithi173 ceph-mon[32402]: osd.11 [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] boot 2024-04-24T23:59:14.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:13 smithi173 ceph-mon[32402]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T23:59:14.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:13 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:59:14.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:13 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4011751738' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:59:14.214 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1714003152,"num_in_osds":12,"osd_in_since":1714003132,"num_remapped_pgs":0} 2024-04-24T23:59:14.215 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:59:14.455 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:59:14.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:59:14.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:14.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:14.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:14.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:14.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:14.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:14.839 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:14.840 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:14.840 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:14.840 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:14 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:14 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:59:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:59:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:15.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:15.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:14 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:15.965 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:15 smithi110 ceph-mon[29187]: Detected new or changed devices on smithi188 2024-04-24T23:59:15.965 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:15 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi188 to 4272M 2024-04-24T23:59:15.965 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:15 smithi110 ceph-mon[29187]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:15 smithi188 ceph-mon[33285]: Detected new or changed devices on smithi188 2024-04-24T23:59:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:15 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi188 to 4272M 2024-04-24T23:59:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:15 smithi188 ceph-mon[33285]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:15 smithi173 ceph-mon[32402]: Detected new or changed devices on smithi188 2024-04-24T23:59:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:15 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi188 to 4272M 2024-04-24T23:59:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:15 smithi173 ceph-mon[32402]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:16.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 -- 172.21.15.110:0/2535331494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d81082b0 msgr2=0x7f11d8108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 --2- 172.21.15.110:0/2535331494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d81082b0 0x7f11d8108690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009990 tx=0x7f11d0009ca0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 -- 172.21.15.110:0/2535331494 shutdown_connections 2024-04-24T23:59:16.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 --2- 172.21.15.110:0/2535331494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d806a8e0 0x7f11d810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 --2- 172.21.15.110:0/2535331494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d81082b0 0x7f11d8108690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 --2- 172.21.15.110:0/2535331494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11d8069f40 0x7f11d806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 -- 172.21.15.110:0/2535331494 >> 172.21.15.110:0/2535331494 conn(0x7f11d80780e0 msgr2=0x7f11d80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 -- 172.21.15.110:0/2535331494 shutdown_connections 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.192+0000 7f11dddbd700 1 -- 172.21.15.110:0/2535331494 wait complete. 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11dddbd700 1 Processor -- start 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11dddbd700 1 -- start start 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11dddbd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d8069f40 0x7f11d819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11dddbd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11d806a8e0 0x7f11d819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11dddbd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d81082b0 0x7f11d8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11dddbd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8112420 con 0x7f11d8069f40 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11dddbd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81122a0 con 0x7f11d806a8e0 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11dddbd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81125a0 con 0x7f11d81082b0 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11d77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d8069f40 0x7f11d819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11d77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d8069f40 0x7f11d819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37478/0 (socket says 172.21.15.110:37478) 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11d77fe700 1 -- 172.21.15.110:0/4045589271 learned_addr learned my addr 172.21.15.110:0/4045589271 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.193+0000 7f11d6ffd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11d806a8e0 0x7f11d819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d77fe700 1 -- 172.21.15.110:0/4045589271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d81082b0 msgr2=0x7f11d8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d7fff700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d81082b0 0x7f11d8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d77fe700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d81082b0 0x7f11d8198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d77fe700 1 -- 172.21.15.110:0/4045589271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11d806a8e0 msgr2=0x7f11d819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d77fe700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11d806a8e0 0x7f11d819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d77fe700 1 -- 172.21.15.110:0/4045589271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d8199000 con 0x7f11d8069f40 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d77fe700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d8069f40 0x7f11d819e6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11d0000c00 tx=0x7f11d0003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d4ff9700 1 -- 172.21.15.110:0/4045589271 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0027070 con 0x7f11d8069f40 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d4ff9700 1 -- 172.21.15.110:0/4045589271 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d000bdc0 con 0x7f11d8069f40 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d8199290 con 0x7f11d8069f40 2024-04-24T23:59:16.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11d4ff9700 1 -- 172.21.15.110:0/4045589271 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0022d30 con 0x7f11d8069f40 2024-04-24T23:59:16.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.194+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d81a5500 con 0x7f11d8069f40 2024-04-24T23:59:16.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.195+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d8103960 con 0x7f11d8069f40 2024-04-24T23:59:16.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.196+0000 7f11d4ff9700 1 -- 172.21.15.110:0/4045589271 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f11d002a070 con 0x7f11d8069f40 2024-04-24T23:59:16.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.197+0000 7f11d4ff9700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f11c00a2c20 0x7f11c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:16.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.197+0000 7f11d6ffd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f11c00a2c20 0x7f11c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:16.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.197+0000 7f11d4ff9700 1 -- 172.21.15.110:0/4045589271 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f11d00cf910 con 0x7f11d8069f40 2024-04-24T23:59:16.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.197+0000 7f11d6ffd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f11c00a2c20 0x7f11c00a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009550 tx=0x7f11cc0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:16.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.201+0000 7f11d4ff9700 1 -- 172.21.15.110:0/4045589271 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f11d0013070 con 0x7f11d8069f40 2024-04-24T23:59:16.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.400+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11d8199d50 con 0x7f11d8069f40 2024-04-24T23:59:16.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.401+0000 7f11d4ff9700 1 -- 172.21.15.110:0/4045589271 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18504 (secure 0 0 0) 0x7f11d009cda0 con 0x7f11d8069f40 2024-04-24T23:59:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:16.401 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":81,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-24T23:59:13.658286+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:59:16.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f11c00a2c20 msgr2=0x7f11c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f11c00a2c20 0x7f11c00a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009550 tx=0x7f11cc0091b0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d8069f40 msgr2=0x7f11d819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:16.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d8069f40 0x7f11d819e6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11d0000c00 tx=0x7f11d0003a80 comp rx=0 tx=0).stop 2024-04-24T23:59:16.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 shutdown_connections 2024-04-24T23:59:16.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f11c00a2c20 0x7f11c00a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d81082b0 0x7f11d8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11d8069f40 0x7f11d819e6a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 --2- 172.21.15.110:0/4045589271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11d806a8e0 0x7f11d819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:16.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 >> 172.21.15.110:0/4045589271 conn(0x7f11d80780e0 msgr2=0x7f11d80ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:16.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 shutdown_connections 2024-04-24T23:59:16.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:16.403+0000 7f11dddbd700 1 -- 172.21.15.110:0/4045589271 wait complete. 2024-04-24T23:59:16.988 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:16 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4045589271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:59:17.023 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:16 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4045589271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:59:17.052 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:59:17.052 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T23:59:17.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:16 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4045589271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:59:17.286 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:59:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:17 smithi188 ceph-mon[33285]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:17 smithi110 ceph-mon[29187]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:17 smithi173 ceph-mon[32402]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:19.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 -- 172.21.15.110:0/1921123206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c101b90 msgr2=0x7f7a8c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1921123206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c101b90 0x7f7a8c101f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0099c0 tx=0x7f7a7c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 -- 172.21.15.110:0/1921123206 shutdown_connections 2024-04-24T23:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1921123206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a8c1064b0 0x7f7a8c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1921123206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c101b90 0x7f7a8c101f70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1921123206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8c102540 0x7f7a8c105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 -- 172.21.15.110:0/1921123206 >> 172.21.15.110:0/1921123206 conn(0x7f7a8c0777a0 msgr2=0x7f7a8c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 -- 172.21.15.110:0/1921123206 shutdown_connections 2024-04-24T23:59:19.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.122+0000 7f7a9089d700 1 -- 172.21.15.110:0/1921123206 wait complete. 2024-04-24T23:59:19.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.123+0000 7f7a9089d700 1 Processor -- start 2024-04-24T23:59:19.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.123+0000 7f7a9089d700 1 -- start start 2024-04-24T23:59:19.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.123+0000 7f7a9089d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a8c101b90 0x7f7a8c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.123+0000 7f7a9089d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c102540 0x7f7a8c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.123+0000 7f7a9089d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8c1064b0 0x7f7a8c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.123+0000 7f7a9089d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c111480 con 0x7f7a8c102540 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.123+0000 7f7a9089d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c111300 con 0x7f7a8c1064b0 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.123+0000 7f7a9089d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c111600 con 0x7f7a8c101b90 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c102540 0x7f7a8c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c102540 0x7f7a8c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45832/0 (socket says 172.21.15.110:45832) 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 -- 172.21.15.110:0/1144284588 learned_addr learned my addr 172.21.15.110:0/1144284588 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a8a59c700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a8c101b90 0x7f7a8c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a8ad9d700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8c1064b0 0x7f7a8c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 -- 172.21.15.110:0/1144284588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a8c101b90 msgr2=0x7f7a8c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a8c101b90 0x7f7a8c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 -- 172.21.15.110:0/1144284588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8c1064b0 msgr2=0x7f7a8c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:19.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8c1064b0 0x7f7a8c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 -- 172.21.15.110:0/1144284588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a6f00 con 0x7f7a8c102540 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a8ad9d700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8c1064b0 0x7f7a8c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a89d9b700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c102540 0x7f7a8c197c30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000b500 tx=0x7f7a8000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a7b7fe700 1 -- 172.21.15.110:0/1144284588 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80015020 con 0x7f7a8c102540 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a7b7fe700 1 -- 172.21.15.110:0/1144284588 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a80011ce0 con 0x7f7a8c102540 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a71f0 con 0x7f7a8c102540 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a7b7fe700 1 -- 172.21.15.110:0/1144284588 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80010d70 con 0x7f7a8c102540 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.124+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a7780 con 0x7f7a8c102540 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.125+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c109740 con 0x7f7a8c102540 2024-04-24T23:59:19.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.126+0000 7f7a7b7fe700 1 -- 172.21.15.110:0/1144284588 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7a80018020 con 0x7f7a8c102540 2024-04-24T23:59:19.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.127+0000 7f7a7b7fe700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a740a2ca0 0x7f7a740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:19.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.127+0000 7f7a8a59c700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a740a2ca0 0x7f7a740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:19.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.127+0000 7f7a7b7fe700 1 -- 172.21.15.110:0/1144284588 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7a800ce300 con 0x7f7a8c102540 2024-04-24T23:59:19.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.128+0000 7f7a8a59c700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a740a2ca0 0x7f7a740a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009fd0 tx=0x7f7a7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:19.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.132+0000 7f7a7b7fe700 1 -- 172.21.15.110:0/1144284588 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7a8001f020 con 0x7f7a8c102540 2024-04-24T23:59:19.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.324+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a8c102980 con 0x7f7a8c102540 2024-04-24T23:59:19.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.324+0000 7f7a7b7fe700 1 -- 172.21.15.110:0/1144284588 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f7a8009b700 con 0x7f7a8c102540 2024-04-24T23:59:19.325 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a740a2ca0 msgr2=0x7f7a740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a740a2ca0 0x7f7a740a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009fd0 tx=0x7f7a7c005fd0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c102540 msgr2=0x7f7a8c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c102540 0x7f7a8c197c30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000b500 tx=0x7f7a8000b8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 shutdown_connections 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a740a2ca0 0x7f7a740a5160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8c1064b0 0x7f7a8c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a8c101b90 0x7f7a8c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 --2- 172.21.15.110:0/1144284588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a8c102540 0x7f7a8c197c30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.326+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 >> 172.21.15.110:0/1144284588 conn(0x7f7a8c0777a0 msgr2=0x7f7a8c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.327+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 shutdown_connections 2024-04-24T23:59:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:19.327+0000 7f7a9089d700 1 -- 172.21.15.110:0/1144284588 wait complete. 2024-04-24T23:59:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:19 smithi188 ceph-mon[33285]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:19 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1144284588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:59:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:19 smithi110 ceph-mon[29187]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:19 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1144284588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:59:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:19 smithi173 ceph-mon[32402]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:19 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1144284588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:59:20.146 INFO:tasks.cephadm:Adding mds.a on smithi110 2024-04-24T23:59:20.146 INFO:tasks.cephadm:Adding mds.d on smithi110 2024-04-24T23:59:20.146 INFO:tasks.cephadm:Adding mds.g on smithi110 2024-04-24T23:59:20.146 INFO:tasks.cephadm:Adding mds.j on smithi110 2024-04-24T23:59:20.147 INFO:tasks.cephadm:Adding mds.b on smithi173 2024-04-24T23:59:20.147 INFO:tasks.cephadm:Adding mds.e on smithi173 2024-04-24T23:59:20.147 INFO:tasks.cephadm:Adding mds.h on smithi173 2024-04-24T23:59:20.147 INFO:tasks.cephadm:Adding mds.k on smithi173 2024-04-24T23:59:20.147 INFO:tasks.cephadm:Adding mds.c on smithi188 2024-04-24T23:59:20.147 INFO:tasks.cephadm:Adding mds.f on smithi188 2024-04-24T23:59:20.147 INFO:tasks.cephadm:Adding mds.i on smithi188 2024-04-24T23:59:20.147 INFO:tasks.cephadm:Adding mds.l on smithi188 2024-04-24T23:59:20.147 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l' 2024-04-24T23:59:20.398 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:59:21.833 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:21 smithi188 ceph-mon[33285]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:21 smithi110 ceph-mon[29187]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:22.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.064+0000 7fdfd486d700 1 -- 172.21.15.188:0/1437560956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0102290 msgr2=0x7fdfd01026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:22.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.064+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1437560956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0102290 0x7fdfd01026b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009990 tx=0x7fdfc4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.065+0000 7fdfd486d700 1 -- 172.21.15.188:0/1437560956 shutdown_connections 2024-04-24T23:59:22.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.065+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1437560956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd0103e30 0x7fdfd010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.065+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1437560956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0102290 0x7fdfd01026b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.065+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1437560956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd0103490 0x7fdfd01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.065+0000 7fdfd486d700 1 -- 172.21.15.188:0/1437560956 >> 172.21.15.188:0/1437560956 conn(0x7fdfd00fd830 msgr2=0x7fdfd00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:22.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.065+0000 7fdfd486d700 1 -- 172.21.15.188:0/1437560956 shutdown_connections 2024-04-24T23:59:22.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.065+0000 7fdfd486d700 1 -- 172.21.15.188:0/1437560956 wait complete. 2024-04-24T23:59:22.067 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfd486d700 1 Processor -- start 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfd486d700 1 -- start start 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfd486d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd0102290 0x7fdfd0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfd486d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0103490 0x7fdfd01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfd486d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd0103e30 0x7fdfd019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfd486d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd010dd80 con 0x7fdfd0103e30 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfd486d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd010dc00 con 0x7fdfd0102290 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfd486d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd010df00 con 0x7fdfd0103490 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0103490 0x7fdfd01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0103490 0x7fdfd01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:42048/0 (socket says 172.21.15.188:42048) 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 -- 172.21.15.188:0/1096313599 learned_addr learned my addr 172.21.15.188:0/1096313599 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfce59c700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd0102290 0x7fdfd0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 -- 172.21.15.188:0/1096313599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd0102290 msgr2=0x7fdfd0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:22.068 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd0102290 0x7fdfd0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfced9d700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd0103e30 0x7fdfd019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 -- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd0103e30 msgr2=0x7fdfd019fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd0103e30 0x7fdfd019fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 -- 172.21.15.188:0/1096313599 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd01a00e0 con 0x7fdfd0103490 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfce59c700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd0102290 0x7fdfd0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.066+0000 7fdfcdd9b700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0103490 0x7fdfd01989c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000c950 tx=0x7fdfc000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.067+0000 7fdfbf7fe700 1 -- 172.21.15.188:0/1096313599 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0007ac0 con 0x7fdfd0103490 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.067+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd01a03d0 con 0x7fdfd0103490 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.067+0000 7fdfbf7fe700 1 -- 172.21.15.188:0/1096313599 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfc0004ce0 con 0x7fdfd0103490 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.067+0000 7fdfbf7fe700 1 -- 172.21.15.188:0/1096313599 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc00055e0 con 0x7fdfd0103490 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.067+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd01a0990 con 0x7fdfd0103490 2024-04-24T23:59:22.069 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.068+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd0068f20 con 0x7fdfd0103490 2024-04-24T23:59:22.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.069+0000 7fdfbf7fe700 1 -- 172.21.15.188:0/1096313599 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdfc0004750 con 0x7fdfd0103490 2024-04-24T23:59:22.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.069+0000 7fdfbf7fe700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdfb80a2c20 0x7fdfb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:22.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.069+0000 7fdfbf7fe700 1 -- 172.21.15.188:0/1096313599 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdfc00ce850 con 0x7fdfd0103490 2024-04-24T23:59:22.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.069+0000 7fdfce59c700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdfb80a2c20 0x7fdfb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:22.070 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.070+0000 7fdfce59c700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdfb80a2c20 0x7fdfb80a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4003920 tx=0x7fdfc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:22.073 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.074+0000 7fdfbf7fe700 1 -- 172.21.15.188:0/1096313599 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdfc0014030 con 0x7fdfd0103490 2024-04-24T23:59:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:21 smithi173 ceph-mon[32402]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:22.281 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.280+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}) -- 0x7fdfd0062870 con 0x7fdfb80a2c20 2024-04-24T23:59:22.289 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.289+0000 7fdfbf7fe700 1 -- 172.21.15.188:0/1096313599 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdfd0062870 con 0x7fdfb80a2c20 2024-04-24T23:59:22.289 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mds.all update... 2024-04-24T23:59:22.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdfb80a2c20 msgr2=0x7fdfb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:22.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdfb80a2c20 0x7fdfb80a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4003920 tx=0x7fdfc4005fb0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0103490 msgr2=0x7fdfd01989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:22.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0103490 0x7fdfd01989c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000c950 tx=0x7fdfc000cd10 comp rx=0 tx=0).stop 2024-04-24T23:59:22.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 shutdown_connections 2024-04-24T23:59:22.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdfb80a2c20 0x7fdfb80a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdfd0103e30 0x7fdfd019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.291 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfd0102290 0x7fdfd0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.292 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 --2- 172.21.15.188:0/1096313599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd0103490 0x7fdfd01989c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:22.292 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 >> 172.21.15.188:0/1096313599 conn(0x7fdfd00fd830 msgr2=0x7fdfd01068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:22.292 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 shutdown_connections 2024-04-24T23:59:22.292 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-24T23:59:22.291+0000 7fdfd486d700 1 -- 172.21.15.188:0/1096313599 wait complete. 2024-04-24T23:59:23.003 DEBUG:teuthology.orchestra.run.smithi110:mds.a> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T23:59:23.006 DEBUG:teuthology.orchestra.run.smithi110:mds.d> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T23:59:23.049 DEBUG:teuthology.orchestra.run.smithi110:mds.g> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T23:59:23.093 DEBUG:teuthology.orchestra.run.smithi110:mds.j> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T23:59:23.136 DEBUG:teuthology.orchestra.run.smithi173:mds.b> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T23:59:23.138 DEBUG:teuthology.orchestra.run.smithi173:mds.e> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T23:59:23.182 DEBUG:teuthology.orchestra.run.smithi173:mds.h> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T23:59:23.225 DEBUG:teuthology.orchestra.run.smithi173:mds.k> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T23:59:23.269 DEBUG:teuthology.orchestra.run.smithi188:mds.c> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T23:59:23.271 DEBUG:teuthology.orchestra.run.smithi188:mds.f> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T23:59:23.314 DEBUG:teuthology.orchestra.run.smithi188:mds.i> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T23:59:23.357 DEBUG:teuthology.orchestra.run.smithi188:mds.l> sudo journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T23:59:23.399 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T23:59:23.400 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:59:23.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 -- 172.21.15.110:0/3847477414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0101880 msgr2=0x7fcea0101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:23.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/3847477414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0101880 0x7fcea0101d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fce88009900 tx=0x7fce8802a410 comp rx=0 tx=0).stop 2024-04-24T23:59:23.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 -- 172.21.15.110:0/3847477414 shutdown_connections 2024-04-24T23:59:23.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/3847477414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea0102240 0x7fcea0108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/3847477414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0101880 0x7fcea0101d00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/3847477414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea0100690 0x7fcea0100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 -- 172.21.15.110:0/3847477414 >> 172.21.15.110:0/3847477414 conn(0x7fcea00fbc20 msgr2=0x7fcea00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 -- 172.21.15.110:0/3847477414 shutdown_connections 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.515+0000 7fcea4c1c640 1 -- 172.21.15.110:0/3847477414 wait complete. 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fcea4c1c640 1 Processor -- start 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fcea4c1c640 1 -- start start 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fcea4c1c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0100690 0x7fcea0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fcea4c1c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea0101880 0x7fcea0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fcea4c1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea0102240 0x7fcea019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fcea4c1c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcea010d2e0 con 0x7fcea0100690 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fcea4c1c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcea010d160 con 0x7fcea0101880 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fcea4c1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcea010d460 con 0x7fcea0102240 2024-04-24T23:59:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0100690 0x7fcea0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0100690 0x7fcea0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45846/0 (socket says 172.21.15.110:45846) 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9e59c640 1 -- 172.21.15.110:0/1571796340 learned_addr learned my addr 172.21.15.110:0/1571796340 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9ed9d640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea0102240 0x7fcea019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce97fff640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea0101880 0x7fcea0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9e59c640 1 -- 172.21.15.110:0/1571796340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea0102240 msgr2=0x7fcea019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9e59c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea0102240 0x7fcea019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9e59c640 1 -- 172.21.15.110:0/1571796340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea0101880 msgr2=0x7fcea0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9e59c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea0101880 0x7fcea0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce9e59c640 1 -- 172.21.15.110:0/1571796340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea019f640 con 0x7fcea0100690 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.516+0000 7fce97fff640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea0101880 0x7fcea0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.517+0000 7fce9e59c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0100690 0x7fcea0197a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fce9000d770 tx=0x7fce9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.517+0000 7fce977fe640 1 -- 172.21.15.110:0/1571796340 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90017070 con 0x7fcea0100690 2024-04-24T23:59:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.517+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea019f930 con 0x7fcea0100690 2024-04-24T23:59:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.517+0000 7fce977fe640 1 -- 172.21.15.110:0/1571796340 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce90009c30 con 0x7fcea0100690 2024-04-24T23:59:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.517+0000 7fce977fe640 1 -- 172.21.15.110:0/1571796340 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90003d00 con 0x7fcea0100690 2024-04-24T23:59:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.517+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea019fed0 con 0x7fcea0100690 2024-04-24T23:59:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.518+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea0107a00 con 0x7fcea0100690 2024-04-24T23:59:23.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.519+0000 7fce977fe640 1 -- 172.21.15.110:0/1571796340 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fce90003ea0 con 0x7fcea0100690 2024-04-24T23:59:23.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.519+0000 7fce977fe640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce8c0a2b90 0x7fce8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:23.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.520+0000 7fce97fff640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce8c0a2b90 0x7fce8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:23.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.520+0000 7fce977fe640 1 -- 172.21.15.110:0/1571796340 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fce900ce8c0 con 0x7fcea0100690 2024-04-24T23:59:23.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.520+0000 7fce97fff640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce8c0a2b90 0x7fce8c0a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcea0198f30 tx=0x7fce88032c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:23.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.524+0000 7fce977fe640 1 -- 172.21.15.110:0/1571796340 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce9009bcc0 con 0x7fcea0100690 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: Saving service mds.all spec with placement smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:23.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:23 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:23.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.701+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcea0062340 con 0x7fcea0100690 2024-04-24T23:59:23.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.702+0000 7fce977fe640 1 -- 172.21.15.110:0/1571796340 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18504 (secure 0 0 0) 0x7fce90009de0 con 0x7fcea0100690 2024-04-24T23:59:23.703 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:23.703 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":81,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-24T23:59:13.658286+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:59:23.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce8c0a2b90 msgr2=0x7fce8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:23.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce8c0a2b90 0x7fce8c0a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcea0198f30 tx=0x7fce88032c40 comp rx=0 tx=0).stop 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0100690 msgr2=0x7fcea0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0100690 0x7fcea0197a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fce9000d770 tx=0x7fce9000dc30 comp rx=0 tx=0).stop 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 shutdown_connections 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce8c0a2b90 0x7fce8c0a5050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcea0102240 0x7fcea019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea0101880 0x7fcea0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 --2- 172.21.15.110:0/1571796340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcea0100690 0x7fcea0197a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 >> 172.21.15.110:0/1571796340 conn(0x7fcea00fbc20 msgr2=0x7fcea0104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 shutdown_connections 2024-04-24T23:59:23.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.704+0000 7fcea4c1c640 1 -- 172.21.15.110:0/1571796340 wait complete. 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: Saving service mds.all spec with placement smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:23.715 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:23.716 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:23 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:23.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:59:23.717 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:59:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: Saving service mds.all spec with placement smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:23 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:23.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 -- 172.21.15.110:0/4078769319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800100680 msgr2=0x7ff800100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 --2- 172.21.15.110:0/4078769319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800100680 0x7ff800100aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009ca0 tx=0x7ff7f002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 -- 172.21.15.110:0/4078769319 shutdown_connections 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 --2- 172.21.15.110:0/4078769319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff800102230 0x7ff800108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 --2- 172.21.15.110:0/4078769319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff800101870 0x7ff800101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 --2- 172.21.15.110:0/4078769319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800100680 0x7ff800100aa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 -- 172.21.15.110:0/4078769319 >> 172.21.15.110:0/4078769319 conn(0x7ff8000fbbf0 msgr2=0x7ff8000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 -- 172.21.15.110:0/4078769319 shutdown_connections 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.818+0000 7ff80551d640 1 -- 172.21.15.110:0/4078769319 wait complete. 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff80551d640 1 Processor -- start 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff80551d640 1 -- start start 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff80551d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff800100680 0x7ff800197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff80551d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff800101870 0x7ff800198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff80551d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800102230 0x7ff80019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:23.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff80551d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff80010c1b0 con 0x7ff800102230 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff80551d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff80010c030 con 0x7ff800101870 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff80551d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff80010c330 con 0x7ff800100680 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7ff7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800102230 0x7ff80019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7ff7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800102230 0x7ff80019f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45872/0 (socket says 172.21.15.110:45872) 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7ff7fe640 1 -- 172.21.15.110:0/280359839 learned_addr learned my addr 172.21.15.110:0/280359839 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7feffd640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff800100680 0x7ff800197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7fe7fc640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff800101870 0x7ff800198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7ff7fe640 1 -- 172.21.15.110:0/280359839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff800100680 msgr2=0x7ff800197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7ff7fe640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff800100680 0x7ff800197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7ff7fe640 1 -- 172.21.15.110:0/280359839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff800101870 msgr2=0x7ff800198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7ff7fe640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff800101870 0x7ff800198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7ff7fe640 1 -- 172.21.15.110:0/280359839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80019f6d0 con 0x7ff800102230 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.819+0000 7ff7fe7fc640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff800101870 0x7ff800198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:23.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.820+0000 7ff7ff7fe640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800102230 0x7ff80019f060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800c970 tx=0x7ff7f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:23.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.820+0000 7ff7e7fff640 1 -- 172.21.15.110:0/280359839 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8007bb0 con 0x7ff800102230 2024-04-24T23:59:23.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.820+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80019f9c0 con 0x7ff800102230 2024-04-24T23:59:23.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.820+0000 7ff7e7fff640 1 -- 172.21.15.110:0/280359839 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7f8007d50 con 0x7ff800102230 2024-04-24T23:59:23.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.820+0000 7ff7e7fff640 1 -- 172.21.15.110:0/280359839 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004e10 con 0x7ff800102230 2024-04-24T23:59:23.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.820+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80019ff10 con 0x7ff800102230 2024-04-24T23:59:23.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.821+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8001a01a0 con 0x7ff800102230 2024-04-24T23:59:23.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.822+0000 7ff7e7fff640 1 -- 172.21.15.110:0/280359839 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff7f8005020 con 0x7ff800102230 2024-04-24T23:59:23.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.822+0000 7ff7e7fff640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff7e80a2b90 0x7ff7e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:23.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.823+0000 7ff7feffd640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff7e80a2b90 0x7ff7e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:23.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.823+0000 7ff7e7fff640 1 -- 172.21.15.110:0/280359839 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff7f80ce920 con 0x7ff800102230 2024-04-24T23:59:23.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.823+0000 7ff7feffd640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff7e80a2b90 0x7ff7e80a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000f80 tx=0x7ff7f0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:23.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:23.827+0000 7ff7e7fff640 1 -- 172.21.15.110:0/280359839 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff7f8014030 con 0x7ff800102230 2024-04-24T23:59:24.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.001+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8001a0370 con 0x7ff800102230 2024-04-24T23:59:24.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.001+0000 7ff7e7fff640 1 -- 172.21.15.110:0/280359839 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7ff7f809bd20 con 0x7ff800102230 2024-04-24T23:59:24.001 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-24T23:59:24.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff7e80a2b90 msgr2=0x7ff7e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:24.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff7e80a2b90 0x7ff7e80a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000f80 tx=0x7ff7f0004a40 comp rx=0 tx=0).stop 2024-04-24T23:59:24.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800102230 msgr2=0x7ff80019f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:24.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800102230 0x7ff80019f060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800c970 tx=0x7ff7f800ce30 comp rx=0 tx=0).stop 2024-04-24T23:59:24.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 shutdown_connections 2024-04-24T23:59:24.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff7e80a2b90 0x7ff7e80a5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff800102230 0x7ff80019f060 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff800101870 0x7ff800198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 --2- 172.21.15.110:0/280359839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff800100680 0x7ff800197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 >> 172.21.15.110:0/280359839 conn(0x7ff8000fbbf0 msgr2=0x7ff8000688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:24.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 shutdown_connections 2024-04-24T23:59:24.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.003+0000 7ff80551d640 1 -- 172.21.15.110:0/280359839 wait complete. 2024-04-24T23:59:24.014 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:24.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.111+0000 7f21a6d9e640 1 -- 172.21.15.110:0/2802676663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a0102070 msgr2=0x7f21a0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:24.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.111+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/2802676663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a0102070 0x7f21a0108900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2190009900 tx=0x7f219002a410 comp rx=0 tx=0).stop 2024-04-24T23:59:24.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.111+0000 7f21a6d9e640 1 -- 172.21.15.110:0/2802676663 shutdown_connections 2024-04-24T23:59:24.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.111+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/2802676663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a0102070 0x7f21a0108900 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.111+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/2802676663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a01016b0 0x7f21a0101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.111+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/2802676663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a01004c0 0x7f21a01008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 -- 172.21.15.110:0/2802676663 >> 172.21.15.110:0/2802676663 conn(0x7f21a00fba50 msgr2=0x7f21a00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:24.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 -- 172.21.15.110:0/2802676663 shutdown_connections 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 -- 172.21.15.110:0/2802676663 wait complete. 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 Processor -- start 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 -- start start 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a01004c0 0x7f21a0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a01016b0 0x7f21a0197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a0102070 0x7f21a019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010bfd0 con 0x7f21a01004c0 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010be50 con 0x7f21a0102070 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.112+0000 7f21a6d9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f21a010c150 con 0x7f21a01016b0 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a01004c0 0x7f21a0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a01004c0 0x7f21a0197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45884/0 (socket says 172.21.15.110:45884) 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 -- 172.21.15.110:0/1332590341 learned_addr learned my addr 172.21.15.110:0/1332590341 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a559b640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a01016b0 0x7f21a0197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:24.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a659d640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a0102070 0x7f21a019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 -- 172.21.15.110:0/1332590341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a01016b0 msgr2=0x7f21a0197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a01016b0 0x7f21a0197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 -- 172.21.15.110:0/1332590341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a0102070 msgr2=0x7f21a019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a0102070 0x7f21a019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 -- 172.21.15.110:0/1332590341 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a019f3b0 con 0x7f21a01004c0 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a659d640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a0102070 0x7f21a019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a5d9c640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a01004c0 0x7f21a0197850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f219800ebc0 tx=0x7f219800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f219effd640 1 -- 172.21.15.110:0/1332590341 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198003b70 con 0x7f21a01004c0 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f219effd640 1 -- 172.21.15.110:0/1332590341 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2198003d10 con 0x7f21a01004c0 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a019f6a0 con 0x7f21a01004c0 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f219effd640 1 -- 172.21.15.110:0/1332590341 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198019840 con 0x7f21a01004c0 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.113+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a019fbf0 con 0x7f21a01004c0 2024-04-24T23:59:24.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.114+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a019feb0 con 0x7f21a01004c0 2024-04-24T23:59:24.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.115+0000 7f219effd640 1 -- 172.21.15.110:0/1332590341 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f21980199e0 con 0x7f21a01004c0 2024-04-24T23:59:24.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.116+0000 7f219effd640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f21880a2b90 0x7f21880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:24.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.116+0000 7f21a559b640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f21880a2b90 0x7f21880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:24.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.116+0000 7f219effd640 1 -- 172.21.15.110:0/1332590341 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f21980cf5c0 con 0x7f21a01004c0 2024-04-24T23:59:24.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.116+0000 7f21a559b640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f21880a2b90 0x7f21880a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21a0198ca0 tx=0x7f219400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:24.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.119+0000 7f219effd640 1 -- 172.21.15.110:0/1332590341 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f219809ca50 con 0x7f21a01004c0 2024-04-24T23:59:24.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.336+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21a01a0080 con 0x7f21a01004c0 2024-04-24T23:59:24.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.336+0000 7f219effd640 1 -- 172.21.15.110:0/1332590341 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f219809c350 con 0x7f21a01004c0 2024-04-24T23:59:24.336 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:24.336 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T23:59:24.337 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 1 2024-04-24T23:59:24.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f21880a2b90 msgr2=0x7f21880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:24.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f21880a2b90 0x7f21880a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21a0198ca0 tx=0x7f219400a2d0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a01004c0 msgr2=0x7f21a0197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a01004c0 0x7f21a0197850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f219800ebc0 tx=0x7f219800c6a0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 shutdown_connections 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f21880a2b90 0x7f21880a5050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a0102070 0x7f21a019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a01016b0 0x7f21a0197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 --2- 172.21.15.110:0/1332590341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21a01004c0 0x7f21a0197850 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 >> 172.21.15.110:0/1332590341 conn(0x7f21a00fba50 msgr2=0x7f21a00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 shutdown_connections 2024-04-24T23:59:24.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:24.338+0000 7f21a6d9e640 1 -- 172.21.15.110:0/1332590341 wait complete. 2024-04-24T23:59:24.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:24 smithi188 ceph-mon[33285]: Deploying daemon mds.l on smithi188 2024-04-24T23:59:24.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:24 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1571796340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:59:24.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:24 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/280359839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:59:24.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:24 smithi173 ceph-mon[32402]: Deploying daemon mds.l on smithi188 2024-04-24T23:59:24.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:24 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1571796340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:59:24.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:24 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/280359839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:59:24.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:24 smithi110 ceph-mon[29187]: Deploying daemon mds.l on smithi188 2024-04-24T23:59:24.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:24 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1571796340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:59:24.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:24 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/280359839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:59:25.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:25 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1332590341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:25.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:25 smithi188 ceph-mon[33285]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:25.518 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:25 smithi188 systemd[1]: Starting Ceph mds.l for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:25.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:25 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1332590341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:25.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:25 smithi173 ceph-mon[32402]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:25.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:25 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1332590341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:25.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:25 smithi110 ceph-mon[29187]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:25.938 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:25 smithi188 podman[62862]: 2024-04-24 23:59:25.561117467 +0000 UTC m=+0.018597956 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:25.939 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:25 smithi188 podman[62862]: 2024-04-24 23:59:25.679666544 +0000 UTC m=+0.137147023 container create 5a226839493fa2c419b252abdee8b638abf90c4b3604a95deeeef6f3ce132f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T23:59:26.267 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:25 smithi188 podman[62862]: 2024-04-24 23:59:25.93947385 +0000 UTC m=+0.396954334 container init 5a226839493fa2c419b252abdee8b638abf90c4b3604a95deeeef6f3ce132f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:59:26.267 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:25 smithi188 podman[62862]: 2024-04-24 23:59:25.942310771 +0000 UTC m=+0.399791264 container start 5a226839493fa2c419b252abdee8b638abf90c4b3604a95deeeef6f3ce132f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-24T23:59:26.268 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:25 smithi188 ceph-mds[62882]: -- 172.21.15.188:0/2891531141 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55da30f77a40 con 0x55da30f4a800 2024-04-24T23:59:26.268 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l[62878]: starting mds.l at 2024-04-24T23:59:26.268 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:26 smithi188 bash[62862]: 5a226839493fa2c419b252abdee8b638abf90c4b3604a95deeeef6f3ce132f77 2024-04-24T23:59:26.268 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 24 23:59:26 smithi188 systemd[1]: Started Ceph mds.l for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:26.350 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:26.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 -- 172.21.15.110:0/2320784667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 msgr2=0x7f50fc108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:26.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 --2- 172.21.15.110:0/2320784667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 0x7f50fc108ac0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009ca0 tx=0x7f50ec02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 -- 172.21.15.110:0/2320784667 shutdown_connections 2024-04-24T23:59:26.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 --2- 172.21.15.110:0/2320784667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 0x7f50fc108ac0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 --2- 172.21.15.110:0/2320784667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50fc101870 0x7f50fc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 --2- 172.21.15.110:0/2320784667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100680 0x7f50fc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 -- 172.21.15.110:0/2320784667 >> 172.21.15.110:0/2320784667 conn(0x7f50fc0fbbf0 msgr2=0x7f50fc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 -- 172.21.15.110:0/2320784667 shutdown_connections 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.459+0000 7f5103c12640 1 -- 172.21.15.110:0/2320784667 wait complete. 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f5103c12640 1 Processor -- start 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f5103c12640 1 -- start start 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f5103c12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100680 0x7f50fc1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f5103c12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50fc101870 0x7f50fc197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f5103c12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 0x7f50fc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f5103c12640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc10c1b0 con 0x7f50fc102230 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f5103c12640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc10c030 con 0x7f50fc100680 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f5103c12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc10c330 con 0x7f50fc101870 2024-04-24T23:59:26.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 0x7f50fc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51019ae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100680 0x7f50fc1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 0x7f50fc19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45900/0 (socket says 172.21.15.110:45900) 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 -- 172.21.15.110:0/1690095261 learned_addr learned my addr 172.21.15.110:0/1690095261 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51011ad640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50fc101870 0x7f50fc197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 -- 172.21.15.110:0/1690095261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50fc101870 msgr2=0x7f50fc197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50fc101870 0x7f50fc197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 -- 172.21.15.110:0/1690095261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100680 msgr2=0x7f50fc1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100680 0x7f50fc1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 -- 172.21.15.110:0/1690095261 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc19f5d0 con 0x7f50fc102230 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51019ae640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100680 0x7f50fc1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.460+0000 7f51021af640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 0x7f50fc19ef60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f50ec031970 tx=0x7f50ec003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.461+0000 7f50faffd640 1 -- 172.21.15.110:0/1690095261 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec004040 con 0x7f50fc102230 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.461+0000 7f50faffd640 1 -- 172.21.15.110:0/1690095261 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50ec004690 con 0x7f50fc102230 2024-04-24T23:59:26.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.461+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc19f860 con 0x7f50fc102230 2024-04-24T23:59:26.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.461+0000 7f50faffd640 1 -- 172.21.15.110:0/1690095261 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec030c60 con 0x7f50fc102230 2024-04-24T23:59:26.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.461+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc19fe00 con 0x7f50fc102230 2024-04-24T23:59:26.464 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.462+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc1a0030 con 0x7f50fc102230 2024-04-24T23:59:26.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.463+0000 7f50faffd640 1 -- 172.21.15.110:0/1690095261 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f50ec0041e0 con 0x7f50fc102230 2024-04-24T23:59:26.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.463+0000 7f50faffd640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f50e40a2c10 0x7f50e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:26.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.463+0000 7f51019ae640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f50e40a2c10 0x7f50e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:26.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.463+0000 7f50faffd640 1 -- 172.21.15.110:0/1690095261 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f50ec0f0040 con 0x7f50fc102230 2024-04-24T23:59:26.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.464+0000 7f51019ae640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f50e40a2c10 0x7f50e40a50d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f50fc1016d0 tx=0x7f50f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:26.467 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.468+0000 7f50faffd640 1 -- 172.21.15.110:0/1690095261 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f50ec0bd440 con 0x7f50fc102230 2024-04-24T23:59:26.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.679+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50fc1a0200 con 0x7f50fc102230 2024-04-24T23:59:26.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.680+0000 7f50faffd640 1 -- 172.21.15.110:0/1690095261 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f50ec0bcd40 con 0x7f50fc102230 2024-04-24T23:59:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:26.680 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:59:26.681 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 2 2024-04-24T23:59:26.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f50e40a2c10 msgr2=0x7f50e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:26.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f50e40a2c10 0x7f50e40a50d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f50fc1016d0 tx=0x7f50f4005f90 comp rx=0 tx=0).stop 2024-04-24T23:59:26.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 msgr2=0x7f50fc19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:26.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 0x7f50fc19ef60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f50ec031970 tx=0x7f50ec003ba0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 shutdown_connections 2024-04-24T23:59:26.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f50e40a2c10 0x7f50e40a50d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f50fc102230 0x7f50fc19ef60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50fc101870 0x7f50fc197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 --2- 172.21.15.110:0/1690095261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100680 0x7f50fc1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:26.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 >> 172.21.15.110:0/1690095261 conn(0x7f50fc0fbbf0 msgr2=0x7f50fc0688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:26.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 shutdown_connections 2024-04-24T23:59:26.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:26.682+0000 7f5103c12640 1 -- 172.21.15.110:0/1690095261 wait complete. 2024-04-24T23:59:27.047 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.188:6832/2454659277,v1:172.21.15.188:6833/2454659277] up:boot 2024-04-24T23:59:27.047 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: fsmap 1 up:standby 2024-04-24T23:59:27.047 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:59:27.047 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.047 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.047 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.048 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:27.048 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:27.048 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:27.048 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: Deploying daemon mds.k on smithi173 2024-04-24T23:59:27.048 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:27.048 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:26 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1690095261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.188:6832/2454659277,v1:172.21.15.188:6833/2454659277] up:boot 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: fsmap 1 up:standby 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: Deploying daemon mds.k on smithi173 2024-04-24T23:59:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:27.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:26 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1690095261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:27.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.188:6832/2454659277,v1:172.21.15.188:6833/2454659277] up:boot 2024-04-24T23:59:27.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: fsmap 1 up:standby 2024-04-24T23:59:27.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: Deploying daemon mds.k on smithi173 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:27.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:26 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1690095261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:28.602 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 systemd[1]: Starting Ceph mds.k for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:28.694 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:28.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 -- 172.21.15.110:0/498499706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78107460 msgr2=0x7f3e781098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:28.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 --2- 172.21.15.110:0/498499706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78107460 0x7f3e781098b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009ca0 tx=0x7f3e7002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:28.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 -- 172.21.15.110:0/498499706 shutdown_connections 2024-04-24T23:59:28.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 --2- 172.21.15.110:0/498499706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78107460 0x7f3e781098b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:28.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 --2- 172.21.15.110:0/498499706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e78104b30 0x7f3e78106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:28.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 --2- 172.21.15.110:0/498499706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e78102200 0x7f3e781045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:28.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 -- 172.21.15.110:0/498499706 >> 172.21.15.110:0/498499706 conn(0x7f3e780fbbf0 msgr2=0x7f3e780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:28.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 -- 172.21.15.110:0/498499706 shutdown_connections 2024-04-24T23:59:28.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 -- 172.21.15.110:0/498499706 wait complete. 2024-04-24T23:59:28.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.804+0000 7f3e7e241640 1 Processor -- start 2024-04-24T23:59:28.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 -- start start 2024-04-24T23:59:28.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e78102200 0x7f3e78101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:28.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78104b30 0x7f3e781001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:28.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e78107460 0x7f3e781006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:28.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7810d320 con 0x7f3e78104b30 2024-04-24T23:59:28.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7810d1a0 con 0x7f3e78102200 2024-04-24T23:59:28.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7810d4a0 con 0x7f3e78107460 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78104b30 0x7f3e781001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78104b30 0x7f3e781001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38504/0 (socket says 172.21.15.110:38504) 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 -- 172.21.15.110:0/307928015 learned_addr learned my addr 172.21.15.110:0/307928015 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e777fe640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e78102200 0x7f3e78101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e77fff640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e78107460 0x7f3e781006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 -- 172.21.15.110:0/307928015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e78107460 msgr2=0x7f3e781006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e78107460 0x7f3e781006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 -- 172.21.15.110:0/307928015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e78102200 msgr2=0x7f3e78101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e78102200 0x7f3e78101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 -- 172.21.15.110:0/307928015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e78100f80 con 0x7f3e78104b30 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e77fff640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e78107460 0x7f3e781006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e76ffd640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78104b30 0x7f3e781001b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00e9c0 tx=0x7f3e6c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:28.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e74ff9640 1 -- 172.21.15.110:0/307928015 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c00ccc0 con 0x7f3e78104b30 2024-04-24T23:59:28.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e74ff9640 1 -- 172.21.15.110:0/307928015 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e6c0054d0 con 0x7f3e78104b30 2024-04-24T23:59:28.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e74ff9640 1 -- 172.21.15.110:0/307928015 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c010430 con 0x7f3e78104b30 2024-04-24T23:59:28.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e781a1ae0 con 0x7f3e78104b30 2024-04-24T23:59:28.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.805+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e781a2080 con 0x7f3e78104b30 2024-04-24T23:59:28.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.806+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e780fd6e0 con 0x7f3e78104b30 2024-04-24T23:59:28.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.807+0000 7f3e74ff9640 1 -- 172.21.15.110:0/307928015 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3e6c004f30 con 0x7f3e78104b30 2024-04-24T23:59:28.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.808+0000 7f3e74ff9640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e600a2c10 0x7f3e600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:28.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.808+0000 7f3e777fe640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e600a2c10 0x7f3e600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:28.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.808+0000 7f3e74ff9640 1 -- 172.21.15.110:0/307928015 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3e6c0cf220 con 0x7f3e78104b30 2024-04-24T23:59:28.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.809+0000 7f3e777fe640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e600a2c10 0x7f3e600a50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009990 tx=0x7f3e68009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:28.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:28.813+0000 7f3e74ff9640 1 -- 172.21.15.110:0/307928015 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e6c09c620 con 0x7f3e78104b30 2024-04-24T23:59:28.882 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 podman[61921]: 2024-04-24 23:59:28.523197165 +0000 UTC m=+0.018656208 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:28.883 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 podman[61921]: 2024-04-24 23:59:28.638615373 +0000 UTC m=+0.134074406 container create 54140fa474372d057f069f26fccf674dcd6071781731d09d7e6be6eb1814d680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:59:28.883 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 podman[61921]: 2024-04-24 23:59:28.883337341 +0000 UTC m=+0.378796375 container init 54140fa474372d057f069f26fccf674dcd6071781731d09d7e6be6eb1814d680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T23:59:29.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.026+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e7802d370 con 0x7f3e78104b30 2024-04-24T23:59:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.028+0000 7f3e74ff9640 1 -- 172.21.15.110:0/307928015 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f3e6c09bf20 con 0x7f3e78104b30 2024-04-24T23:59:29.028 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:29.028 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:59:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 2 2024-04-24T23:59:29.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e600a2c10 msgr2=0x7f3e600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:29.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e600a2c10 0x7f3e600a50d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009990 tx=0x7f3e68009110 comp rx=0 tx=0).stop 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78104b30 msgr2=0x7f3e781001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78104b30 0x7f3e781001b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00e9c0 tx=0x7f3e6c00edc0 comp rx=0 tx=0).stop 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 shutdown_connections 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e600a2c10 0x7f3e600a50d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e78107460 0x7f3e781006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e78104b30 0x7f3e781001b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 --2- 172.21.15.110:0/307928015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e78102200 0x7f3e78101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 >> 172.21.15.110:0/307928015 conn(0x7f3e780fbbf0 msgr2=0x7f3e78102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 shutdown_connections 2024-04-24T23:59:29.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:29.030+0000 7f3e7e241640 1 -- 172.21.15.110:0/307928015 wait complete. 2024-04-24T23:59:29.352 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 podman[61921]: 2024-04-24 23:59:28.886198684 +0000 UTC m=+0.381657724 container start 54140fa474372d057f069f26fccf674dcd6071781731d09d7e6be6eb1814d680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True) 2024-04-24T23:59:29.352 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 ceph-mds[61940]: -- 172.21.15.173:0/3563829517 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650006ffa40 con 0x5650006d3000 2024-04-24T23:59:29.352 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k[61936]: starting mds.k at 2024-04-24T23:59:29.352 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 bash[61921]: 54140fa474372d057f069f26fccf674dcd6071781731d09d7e6be6eb1814d680 2024-04-24T23:59:29.352 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 24 23:59:28 smithi173 systemd[1]: Started Ceph mds.k for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:29.843 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:29 smithi110 ceph-mon[29187]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:29.843 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:29.844 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:29.844 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:29 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/307928015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:29.844 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:29.844 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:29.844 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:29.844 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:29 smithi173 ceph-mon[32402]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:29 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/307928015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:29 smithi188 ceph-mon[33285]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:29 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/307928015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:30.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:30 smithi173 ceph-mon[32402]: Deploying daemon mds.j on smithi110 2024-04-24T23:59:30.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:30 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6832/373959969,v1:172.21.15.173:6833/373959969] up:boot 2024-04-24T23:59:30.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:30 smithi173 ceph-mon[32402]: fsmap 2 up:standby 2024-04-24T23:59:30.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:30 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:59:30.853 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:30 smithi110 ceph-mon[29187]: Deploying daemon mds.j on smithi110 2024-04-24T23:59:30.853 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:30 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6832/373959969,v1:172.21.15.173:6833/373959969] up:boot 2024-04-24T23:59:30.853 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:30 smithi110 ceph-mon[29187]: fsmap 2 up:standby 2024-04-24T23:59:30.853 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:30 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:59:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:30 smithi188 ceph-mon[33285]: Deploying daemon mds.j on smithi110 2024-04-24T23:59:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:30 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6832/373959969,v1:172.21.15.173:6833/373959969] up:boot 2024-04-24T23:59:31.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:30 smithi188 ceph-mon[33285]: fsmap 2 up:standby 2024-04-24T23:59:31.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:30 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:59:31.042 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 -- 172.21.15.110:0/3332842024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374102240 msgr2=0x7f4374108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 --2- 172.21.15.110:0/3332842024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374102240 0x7f4374108ad0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f436c009ca0 tx=0x7f436c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 -- 172.21.15.110:0/3332842024 shutdown_connections 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 --2- 172.21.15.110:0/3332842024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374102240 0x7f4374108ad0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 --2- 172.21.15.110:0/3332842024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4374101880 0x7f4374101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 --2- 172.21.15.110:0/3332842024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374100690 0x7f4374100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 -- 172.21.15.110:0/3332842024 >> 172.21.15.110:0/3332842024 conn(0x7f43740fbc20 msgr2=0x7f43740fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 -- 172.21.15.110:0/3332842024 shutdown_connections 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 -- 172.21.15.110:0/3332842024 wait complete. 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.199+0000 7f437a4a7640 1 Processor -- start 2024-04-24T23:59:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f437a4a7640 1 -- start start 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f437a4a7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4374100690 0x7f43741968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f437a4a7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374101880 0x7f4374196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f437a4a7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374102240 0x7f437419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f437a4a7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f437410c230 con 0x7f4374101880 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f437a4a7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f437410c0b0 con 0x7f4374100690 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f437a4a7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f437410c3b0 con 0x7f4374102240 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374101880 0x7f4374196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374101880 0x7f4374196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38512/0 (socket says 172.21.15.110:38512) 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 -- 172.21.15.110:0/172720841 learned_addr learned my addr 172.21.15.110:0/172720841 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f4373fff640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4374100690 0x7f43741968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f4378a44640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374102240 0x7f437419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 -- 172.21.15.110:0/172720841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374102240 msgr2=0x7f437419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374102240 0x7f437419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 -- 172.21.15.110:0/172720841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4374100690 msgr2=0x7f43741968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4374100690 0x7f43741968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 -- 172.21.15.110:0/172720841 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437419e500 con 0x7f4374101880 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f436b7fe640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374101880 0x7f4374196e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f435c00e9f0 tx=0x7f435c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f4371ffb640 1 -- 172.21.15.110:0/172720841 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c00ccc0 con 0x7f4374101880 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f4371ffb640 1 -- 172.21.15.110:0/172720841 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f435c0108e0 con 0x7f4374101880 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437419e7f0 con 0x7f4374101880 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.200+0000 7f4371ffb640 1 -- 172.21.15.110:0/172720841 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0187c0 con 0x7f4374101880 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.201+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437419ed40 con 0x7f4374101880 2024-04-24T23:59:31.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.202+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43741079b0 con 0x7f4374101880 2024-04-24T23:59:31.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.203+0000 7f4371ffb640 1 -- 172.21.15.110:0/172720841 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f435c010a90 con 0x7f4374101880 2024-04-24T23:59:31.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.203+0000 7f4371ffb640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f43600a2b90 0x7f43600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:31.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.204+0000 7f4373fff640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f43600a2b90 0x7f43600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:31.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.204+0000 7f4371ffb640 1 -- 172.21.15.110:0/172720841 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f435c0cf010 con 0x7f4374101880 2024-04-24T23:59:31.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.204+0000 7f4373fff640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f43600a2b90 0x7f43600a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43741016e0 tx=0x7f4364009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:31.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.208+0000 7f4371ffb640 1 -- 172.21.15.110:0/172720841 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f435c09c410 con 0x7f4374101880 2024-04-24T23:59:31.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.435+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437402d3c0 con 0x7f4374101880 2024-04-24T23:59:31.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.436+0000 7f4371ffb640 1 -- 172.21.15.110:0/172720841 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f435c09bd10 con 0x7f4374101880 2024-04-24T23:59:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:31.436 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:59:31.436 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 3 2024-04-24T23:59:31.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f43600a2b90 msgr2=0x7f43600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:31.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f43600a2b90 0x7f43600a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43741016e0 tx=0x7f4364009110 comp rx=0 tx=0).stop 2024-04-24T23:59:31.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374101880 msgr2=0x7f4374196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:31.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374101880 0x7f4374196e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f435c00e9f0 tx=0x7f435c00edf0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 shutdown_connections 2024-04-24T23:59:31.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f43600a2b90 0x7f43600a5050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4374102240 0x7f437419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4374101880 0x7f4374196e20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 --2- 172.21.15.110:0/172720841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4374100690 0x7f43741968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:31.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 >> 172.21.15.110:0/172720841 conn(0x7f43740fbc20 msgr2=0x7f4374104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:31.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 shutdown_connections 2024-04-24T23:59:31.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:31.438+0000 7f437a4a7640 1 -- 172.21.15.110:0/172720841 wait complete. 2024-04-24T23:59:31.451 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 systemd[1]: Starting Ceph mds.j for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:31.716 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:31 smithi110 ceph-mon[29187]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:31.716 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:31 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/172720841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:31.716 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 podman[69323]: 2024-04-24 23:59:31.445988172 +0000 UTC m=+0.020551779 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:31.716 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 podman[69323]: 2024-04-24 23:59:31.582165461 +0000 UTC m=+0.156729048 container create 1712dc0d25d90930e6b9176bddd77c9bec20ffe05b8a27e5f5a0ec13bb4b0380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:59:31.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:31 smithi173 ceph-mon[32402]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:31.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:31 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/172720841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:31 smithi188 ceph-mon[33285]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:31 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/172720841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:32.286 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 podman[69323]: 2024-04-24 23:59:31.818806218 +0000 UTC m=+0.393369810 container init 1712dc0d25d90930e6b9176bddd77c9bec20ffe05b8a27e5f5a0ec13bb4b0380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T23:59:32.286 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 podman[69323]: 2024-04-24 23:59:31.821526943 +0000 UTC m=+0.396090542 container start 1712dc0d25d90930e6b9176bddd77c9bec20ffe05b8a27e5f5a0ec13bb4b0380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:59:32.286 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 ceph-mds[69344]: -- 172.21.15.110:0/3952696273 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b0813ba40 con 0x555b0810f000 2024-04-24T23:59:32.286 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j[69340]: starting mds.j at 2024-04-24T23:59:32.286 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 bash[69323]: 1712dc0d25d90930e6b9176bddd77c9bec20ffe05b8a27e5f5a0ec13bb4b0380 2024-04-24T23:59:32.287 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 24 23:59:31 smithi110 systemd[1]: Started Ceph mds.j for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:33.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: Deploying daemon mds.d on smithi110 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.110:6834/1261890335,v1:172.21.15.110:6835/1261890335] up:boot 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: fsmap 3 up:standby 2024-04-24T23:59:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:59:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: Deploying daemon mds.d on smithi110 2024-04-24T23:59:33.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:33.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.110:6834/1261890335,v1:172.21.15.110:6835/1261890335] up:boot 2024-04-24T23:59:33.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: fsmap 3 up:standby 2024-04-24T23:59:33.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: Deploying daemon mds.d on smithi110 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.110:6834/1261890335,v1:172.21.15.110:6835/1261890335] up:boot 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: fsmap 3 up:standby 2024-04-24T23:59:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:59:33.454 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 -- 172.21.15.110:0/1939637816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c101870 msgr2=0x7f086c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 --2- 172.21.15.110:0/1939637816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c101870 0x7f086c101cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f085c009900 tx=0x7f085c02a410 comp rx=0 tx=0).stop 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 -- 172.21.15.110:0/1939637816 shutdown_connections 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 --2- 172.21.15.110:0/1939637816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f086c102230 0x7f086c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 --2- 172.21.15.110:0/1939637816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c101870 0x7f086c101cf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 --2- 172.21.15.110:0/1939637816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f086c100680 0x7f086c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 -- 172.21.15.110:0/1939637816 >> 172.21.15.110:0/1939637816 conn(0x7f086c0fbbf0 msgr2=0x7f086c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 -- 172.21.15.110:0/1939637816 shutdown_connections 2024-04-24T23:59:33.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.556+0000 7f08735f9640 1 -- 172.21.15.110:0/1939637816 wait complete. 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f08735f9640 1 Processor -- start 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f08735f9640 1 -- start start 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f08735f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c100680 0x7f086c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f08735f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f086c101870 0x7f086c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f08735f9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f086c102230 0x7f086c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f08735f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10c2f0 con 0x7f086c100680 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f08735f9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10c170 con 0x7f086c102230 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f08735f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10c470 con 0x7f086c101870 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f0871395640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c100680 0x7f086c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f0871395640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c100680 0x7f086c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38542/0 (socket says 172.21.15.110:38542) 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f0871395640 1 -- 172.21.15.110:0/835671650 learned_addr learned my addr 172.21.15.110:0/835671650 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f0870b94640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f086c101870 0x7f086c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.557+0000 7f0871b96640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f086c102230 0x7f086c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f0871395640 1 -- 172.21.15.110:0/835671650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f086c101870 msgr2=0x7f086c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:33.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f0871395640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f086c101870 0x7f086c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f0871395640 1 -- 172.21.15.110:0/835671650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f086c102230 msgr2=0x7f086c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f0871395640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f086c102230 0x7f086c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f0871395640 1 -- 172.21.15.110:0/835671650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c19f760 con 0x7f086c100680 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f0871395640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c100680 0x7f086c197b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f086800b450 tx=0x7f086800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f08627fc640 1 -- 172.21.15.110:0/835671650 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868015020 con 0x7f086c100680 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f08627fc640 1 -- 172.21.15.110:0/835671650 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0868011d40 con 0x7f086c100680 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086c19fa50 con 0x7f086c100680 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f08627fc640 1 -- 172.21.15.110:0/835671650 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086800f340 con 0x7f086c100680 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.558+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086c19ffa0 con 0x7f086c100680 2024-04-24T23:59:33.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.559+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086c068590 con 0x7f086c100680 2024-04-24T23:59:33.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.560+0000 7f08627fc640 1 -- 172.21.15.110:0/835671650 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0868018020 con 0x7f086c100680 2024-04-24T23:59:33.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.560+0000 7f08627fc640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08580a2b90 0x7f08580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:33.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.560+0000 7f0870b94640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08580a2b90 0x7f08580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:33.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.561+0000 7f08627fc640 1 -- 172.21.15.110:0/835671650 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f08680ce740 con 0x7f086c100680 2024-04-24T23:59:33.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.561+0000 7f0870b94640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08580a2b90 0x7f08580a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f085c000c00 tx=0x7f085c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:33.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.565+0000 7f08627fc640 1 -- 172.21.15.110:0/835671650 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f086c068590 con 0x7f086c100680 2024-04-24T23:59:33.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.779+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f086c1a01c0 con 0x7f086c100680 2024-04-24T23:59:33.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.779+0000 7f08627fc640 1 -- 172.21.15.110:0/835671650 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f086801f020 con 0x7f086c100680 2024-04-24T23:59:33.780 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:33.781 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:59:33.781 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 4 2024-04-24T23:59:33.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08580a2b90 msgr2=0x7f08580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:33.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08580a2b90 0x7f08580a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f085c000c00 tx=0x7f085c0314a0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c100680 msgr2=0x7f086c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:33.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c100680 0x7f086c197b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f086800b450 tx=0x7f086800b910 comp rx=0 tx=0).stop 2024-04-24T23:59:33.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 shutdown_connections 2024-04-24T23:59:33.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08580a2b90 0x7f08580a5050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f086c102230 0x7f086c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f086c101870 0x7f086c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 --2- 172.21.15.110:0/835671650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f086c100680 0x7f086c197b40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:33.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 >> 172.21.15.110:0/835671650 conn(0x7f086c0fbbf0 msgr2=0x7f086c0689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:33.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 shutdown_connections 2024-04-24T23:59:33.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:33.781+0000 7f08735f9640 1 -- 172.21.15.110:0/835671650 wait complete. 2024-04-24T23:59:34.174 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:34 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/835671650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:34 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/835671650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:34.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:34 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/835671650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:34.536 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 systemd[1]: Starting Ceph mds.d for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:34.850 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 podman[69675]: 2024-04-24 23:59:34.464846282 +0000 UTC m=+0.017943259 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:34.850 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 podman[69675]: 2024-04-24 23:59:34.606373255 +0000 UTC m=+0.159470225 container create 9759b01d5621230d7c945f9a307bedc43d4188071d844cf3da80106afb184766 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:59:34.851 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 podman[69675]: 2024-04-24 23:59:34.851117247 +0000 UTC m=+0.404214212 container init 9759b01d5621230d7c945f9a307bedc43d4188071d844cf3da80106afb184766 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:59:35.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:35 smithi188 ceph-mon[33285]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:35.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:35 smithi110 ceph-mon[29187]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:35.286 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 podman[69675]: 2024-04-24 23:59:34.853920991 +0000 UTC m=+0.407017955 container start 9759b01d5621230d7c945f9a307bedc43d4188071d844cf3da80106afb184766 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:59:35.286 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 ceph-mds[69694]: -- 172.21.15.110:0/4207821972 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ef005da40 con 0x556ef0031000 2024-04-24T23:59:35.286 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d[69690]: starting mds.d at 2024-04-24T23:59:35.286 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 bash[69675]: 9759b01d5621230d7c945f9a307bedc43d4188071d844cf3da80106afb184766 2024-04-24T23:59:35.286 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 24 23:59:34 smithi110 systemd[1]: Started Ceph mds.d for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:35.351 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:35 smithi173 ceph-mon[32402]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:35.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:35.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 -- 172.21.15.110:0/4064726411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 msgr2=0x7fad18101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/4064726411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 0x7fad18101cf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad00009900 tx=0x7fad0002a410 comp rx=0 tx=0).stop 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 -- 172.21.15.110:0/4064726411 shutdown_connections 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/4064726411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad18102230 0x7fad18108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/4064726411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 0x7fad18101cf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/4064726411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad18100680 0x7fad18100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 -- 172.21.15.110:0/4064726411 >> 172.21.15.110:0/4064726411 conn(0x7fad180fbbf0 msgr2=0x7fad180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 -- 172.21.15.110:0/4064726411 shutdown_connections 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.907+0000 7fad1f1c1640 1 -- 172.21.15.110:0/4064726411 wait complete. 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1f1c1640 1 Processor -- start 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1f1c1640 1 -- start start 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1f1c1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad18100680 0x7fad1806a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:35.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1f1c1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 0x7fad1819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1f1c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad18102230 0x7fad1819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1f1c1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810c2a0 con 0x7fad18101870 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1f1c1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810c120 con 0x7fad18100680 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1f1c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad1810c420 con 0x7fad18102230 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad17fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 0x7fad1819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad17fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 0x7fad1819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38562/0 (socket says 172.21.15.110:38562) 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad17fff640 1 -- 172.21.15.110:0/2853213131 learned_addr learned my addr 172.21.15.110:0/2853213131 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1cf5d640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad18100680 0x7fad1806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad17fff640 1 -- 172.21.15.110:0/2853213131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad18102230 msgr2=0x7fad1819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1d75e640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad18102230 0x7fad1819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad17fff640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad18102230 0x7fad1819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad17fff640 1 -- 172.21.15.110:0/2853213131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad18100680 msgr2=0x7fad1806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:35.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad17fff640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad18100680 0x7fad1806a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:35.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad17fff640 1 -- 172.21.15.110:0/2853213131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1819f780 con 0x7fad18101870 2024-04-24T23:59:35.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.908+0000 7fad1cf5d640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad18100680 0x7fad1806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:35.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.909+0000 7fad17fff640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 0x7fad1819c850 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad00005940 tx=0x7fad00004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:35.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.909+0000 7fad15ffb640 1 -- 172.21.15.110:0/2853213131 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0003f030 con 0x7fad18101870 2024-04-24T23:59:35.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.909+0000 7fad15ffb640 1 -- 172.21.15.110:0/2853213131 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad00003cb0 con 0x7fad18101870 2024-04-24T23:59:35.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.909+0000 7fad15ffb640 1 -- 172.21.15.110:0/2853213131 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00035070 con 0x7fad18101870 2024-04-24T23:59:35.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.909+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1819f9b0 con 0x7fad18101870 2024-04-24T23:59:35.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.909+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1819ff20 con 0x7fad18101870 2024-04-24T23:59:35.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.910+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad18107ab0 con 0x7fad18101870 2024-04-24T23:59:35.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.911+0000 7fad15ffb640 1 -- 172.21.15.110:0/2853213131 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fad00042030 con 0x7fad18101870 2024-04-24T23:59:35.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.912+0000 7fad15ffb640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fad040a2b90 0x7fad040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:35.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.912+0000 7fad15ffb640 1 -- 172.21.15.110:0/2853213131 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fad000ef700 con 0x7fad18101870 2024-04-24T23:59:35.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.912+0000 7fad1cf5d640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fad040a2b90 0x7fad040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:35.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.912+0000 7fad1cf5d640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fad040a2b90 0x7fad040a5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad181016d0 tx=0x7fad100073a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:35.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:35.916+0000 7fad15ffb640 1 -- 172.21.15.110:0/2853213131 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fad000bcb00 con 0x7fad18101870 2024-04-24T23:59:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.129+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad1802d370 con 0x7fad18101870 2024-04-24T23:59:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.130+0000 7fad15ffb640 1 -- 172.21.15.110:0/2853213131 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7fad000bc400 con 0x7fad18101870 2024-04-24T23:59:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:36.130 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:59:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 5 2024-04-24T23:59:36.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fad040a2b90 msgr2=0x7fad040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:36.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fad040a2b90 0x7fad040a5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad181016d0 tx=0x7fad100073a0 comp rx=0 tx=0).stop 2024-04-24T23:59:36.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 msgr2=0x7fad1819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:36.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 0x7fad1819c850 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad00005940 tx=0x7fad00004430 comp rx=0 tx=0).stop 2024-04-24T23:59:36.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 shutdown_connections 2024-04-24T23:59:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fad040a2b90 0x7fad040a5050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad18102230 0x7fad1819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad18101870 0x7fad1819c850 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 --2- 172.21.15.110:0/2853213131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad18100680 0x7fad1806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 >> 172.21.15.110:0/2853213131 conn(0x7fad180fbbf0 msgr2=0x7fad18104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 shutdown_connections 2024-04-24T23:59:36.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:36.132+0000 7fad1f1c1640 1 -- 172.21.15.110:0/2853213131 wait complete. 2024-04-24T23:59:36.142 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.142 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.110:6836/1723817194,v1:172.21.15.110:6837/1723817194] up:boot 2024-04-24T23:59:36.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: fsmap 4 up:standby 2024-04-24T23:59:36.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:59:36.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:36.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:36.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:36.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:36 smithi110 ceph-mon[29187]: Deploying daemon mds.e on smithi173 2024-04-24T23:59:36.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.110:6836/1723817194,v1:172.21.15.110:6837/1723817194] up:boot 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: fsmap 4 up:standby 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:36 smithi188 ceph-mon[33285]: Deploying daemon mds.e on smithi173 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.110:6836/1723817194,v1:172.21.15.110:6837/1723817194] up:boot 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: fsmap 4 up:standby 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:36 smithi173 ceph-mon[32402]: Deploying daemon mds.e on smithi173 2024-04-24T23:59:37.271 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:37 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2853213131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:37.272 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:37 smithi173 ceph-mon[32402]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:37.272 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 systemd[1]: Starting Ceph mds.e for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:37.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:37 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2853213131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:37.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:37 smithi110 ceph-mon[29187]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:37.380 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:37 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2853213131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:37.380 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:37 smithi188 ceph-mon[33285]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:37.753 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 podman[62237]: 2024-04-24 23:59:37.394268338 +0000 UTC m=+0.018431638 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:37.753 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 podman[62237]: 2024-04-24 23:59:37.508159042 +0000 UTC m=+0.132322348 container create a4b4b4bffc1b70f2a72be2c5db17d27833f82d67ebbe19c46490bf356f02e55d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-24T23:59:37.753 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 podman[62237]: 2024-04-24 23:59:37.727488914 +0000 UTC m=+0.351652210 container init a4b4b4bffc1b70f2a72be2c5db17d27833f82d67ebbe19c46490bf356f02e55d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T23:59:37.753 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 podman[62237]: 2024-04-24 23:59:37.730430172 +0000 UTC m=+0.354593469 container start a4b4b4bffc1b70f2a72be2c5db17d27833f82d67ebbe19c46490bf356f02e55d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T23:59:37.753 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 ceph-mds[62257]: -- 172.21.15.173:0/234135478 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c69e7da40 con 0x558c69e51000 2024-04-24T23:59:38.102 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e[62253]: starting mds.e at 2024-04-24T23:59:38.102 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 bash[62237]: a4b4b4bffc1b70f2a72be2c5db17d27833f82d67ebbe19c46490bf356f02e55d 2024-04-24T23:59:38.102 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 24 23:59:37 smithi173 systemd[1]: Started Ceph mds.e for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:38.144 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:38.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 -- 172.21.15.110:0/2684199629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc101d40 msgr2=0x7f16cc102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:38.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 --2- 172.21.15.110:0/2684199629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc101d40 0x7f16cc102160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009900 tx=0x7f16c402a410 comp rx=0 tx=0).stop 2024-04-24T23:59:38.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 -- 172.21.15.110:0/2684199629 shutdown_connections 2024-04-24T23:59:38.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 --2- 172.21.15.110:0/2684199629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16cc101350 0x7f16cc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 --2- 172.21.15.110:0/2684199629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16cc100990 0x7f16cc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 --2- 172.21.15.110:0/2684199629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc101d40 0x7f16cc102160 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 -- 172.21.15.110:0/2684199629 >> 172.21.15.110:0/2684199629 conn(0x7f16cc0fb420 msgr2=0x7f16cc0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 -- 172.21.15.110:0/2684199629 shutdown_connections 2024-04-24T23:59:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 -- 172.21.15.110:0/2684199629 wait complete. 2024-04-24T23:59:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.255+0000 7f16d10a5640 1 Processor -- start 2024-04-24T23:59:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16d10a5640 1 -- start start 2024-04-24T23:59:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16d10a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc100990 0x7f16cc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16d10a5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16cc101350 0x7f16cc198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16d10a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16cc101d40 0x7f16cc19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16d10a5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10d3e0 con 0x7f16cc100990 2024-04-24T23:59:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16d10a5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10d260 con 0x7f16cc101350 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16d10a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10d560 con 0x7f16cc101d40 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc100990 0x7f16cc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc100990 0x7f16cc197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39862/0 (socket says 172.21.15.110:39862) 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 -- 172.21.15.110:0/4236344697 learned_addr learned my addr 172.21.15.110:0/4236344697 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16ca59c640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16cc101350 0x7f16cc198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cb59e640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16cc101d40 0x7f16cc19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 -- 172.21.15.110:0/4236344697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16cc101d40 msgr2=0x7f16cc19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16cc101d40 0x7f16cc19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 -- 172.21.15.110:0/4236344697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16cc101350 msgr2=0x7f16cc198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16cc101350 0x7f16cc198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 -- 172.21.15.110:0/4236344697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc19f6e0 con 0x7f16cc100990 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.256+0000 7f16cad9d640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc100990 0x7f16cc197ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f16c4004770 tx=0x7f16c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.257+0000 7f16c3fff640 1 -- 172.21.15.110:0/4236344697 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c403f030 con 0x7f16cc100990 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.257+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc19f970 con 0x7f16cc100990 2024-04-24T23:59:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.257+0000 7f16c3fff640 1 -- 172.21.15.110:0/4236344697 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c4003aa0 con 0x7f16cc100990 2024-04-24T23:59:38.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.257+0000 7f16c3fff640 1 -- 172.21.15.110:0/4236344697 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4035070 con 0x7f16cc100990 2024-04-24T23:59:38.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.257+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc19fe60 con 0x7f16cc100990 2024-04-24T23:59:38.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.258+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16cc107b40 con 0x7f16cc100990 2024-04-24T23:59:38.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.259+0000 7f16c3fff640 1 -- 172.21.15.110:0/4236344697 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f16c4042030 con 0x7f16cc100990 2024-04-24T23:59:38.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.259+0000 7f16c3fff640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f16b80a2b90 0x7f16b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:38.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.259+0000 7f16ca59c640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f16b80a2b90 0x7f16b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:38.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.259+0000 7f16c3fff640 1 -- 172.21.15.110:0/4236344697 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f16c40ef2e0 con 0x7f16cc100990 2024-04-24T23:59:38.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.260+0000 7f16ca59c640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f16b80a2b90 0x7f16b80a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f16cc198fd0 tx=0x7f16b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:38.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.264+0000 7f16c3fff640 1 -- 172.21.15.110:0/4236344697 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f16c40bc090 con 0x7f16cc100990 2024-04-24T23:59:38.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.479+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16cc062340 con 0x7f16cc100990 2024-04-24T23:59:38.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.480+0000 7f16c3fff640 1 -- 172.21.15.110:0/4236344697 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f16c40c0390 con 0x7f16cc100990 2024-04-24T23:59:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:38.480 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T23:59:38.480 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 6 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f16b80a2b90 msgr2=0x7f16b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f16b80a2b90 0x7f16b80a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f16cc198fd0 tx=0x7f16b4005f90 comp rx=0 tx=0).stop 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc100990 msgr2=0x7f16cc197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc100990 0x7f16cc197ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f16c4004770 tx=0x7f16c4004270 comp rx=0 tx=0).stop 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 shutdown_connections 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f16b80a2b90 0x7f16b80a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16cc101d40 0x7f16cc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16cc101350 0x7f16cc198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 --2- 172.21.15.110:0/4236344697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16cc100990 0x7f16cc197ac0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:38.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 >> 172.21.15.110:0/4236344697 conn(0x7f16cc0fb420 msgr2=0x7f16cc104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:38.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 shutdown_connections 2024-04-24T23:59:38.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:38.482+0000 7f16d10a5640 1 -- 172.21.15.110:0/4236344697 wait complete. 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: Deploying daemon mds.f on smithi188 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6834/1627462282,v1:172.21.15.173:6835/1627462282] up:boot 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: fsmap 5 up:standby 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:59:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4236344697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:39.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:38 smithi188 ceph-mon[33285]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: Deploying daemon mds.f on smithi188 2024-04-24T23:59:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6834/1627462282,v1:172.21.15.173:6835/1627462282] up:boot 2024-04-24T23:59:39.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: fsmap 5 up:standby 2024-04-24T23:59:39.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:59:39.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4236344697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:39.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:38 smithi110 ceph-mon[29187]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: Deploying daemon mds.f on smithi188 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6834/1627462282,v1:172.21.15.173:6835/1627462282] up:boot 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: fsmap 5 up:standby 2024-04-24T23:59:39.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:59:39.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4236344697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:39.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:38 smithi173 ceph-mon[32402]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:40.268 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 systemd[1]: Starting Ceph mds.f for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:40.494 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:40.578 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 podman[63174]: 2024-04-24 23:59:40.196949379 +0000 UTC m=+0.017807757 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:40.578 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 podman[63174]: 2024-04-24 23:59:40.325450475 +0000 UTC m=+0.146308843 container create 3ceaca1cfe29790e3189b1bb75d8da64a38939364c617dcdb8e39f10eddbbb14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T23:59:40.578 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 podman[63174]: 2024-04-24 23:59:40.578311127 +0000 UTC m=+0.399169492 container init 3ceaca1cfe29790e3189b1bb75d8da64a38939364c617dcdb8e39f10eddbbb14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:59:40.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 -- 172.21.15.110:0/1394160427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650100690 msgr2=0x7f9650100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:40.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 --2- 172.21.15.110:0/1394160427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650100690 0x7f9650100ab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9648009900 tx=0x7f964802a410 comp rx=0 tx=0).stop 2024-04-24T23:59:40.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 -- 172.21.15.110:0/1394160427 shutdown_connections 2024-04-24T23:59:40.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 --2- 172.21.15.110:0/1394160427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9650102240 0x7f9650108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 --2- 172.21.15.110:0/1394160427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9650101880 0x7f9650101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 --2- 172.21.15.110:0/1394160427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650100690 0x7f9650100ab0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 -- 172.21.15.110:0/1394160427 >> 172.21.15.110:0/1394160427 conn(0x7f96500fbc40 msgr2=0x7f96500fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:40.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 -- 172.21.15.110:0/1394160427 shutdown_connections 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 -- 172.21.15.110:0/1394160427 wait complete. 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 Processor -- start 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.605+0000 7f9654f14640 1 -- start start 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f9654f14640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9650100690 0x7f96501047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f9654f14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650101880 0x7f9650102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f9654f14640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9650102240 0x7f9650103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f9654f14640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f965010c1c0 con 0x7f9650101880 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f9654f14640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f965010c040 con 0x7f9650100690 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f9654f14640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f965010c340 con 0x7f9650102240 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650101880 0x7f9650102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650101880 0x7f9650102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39874/0 (socket says 172.21.15.110:39874) 2024-04-24T23:59:40.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 -- 172.21.15.110:0/3939456119 learned_addr learned my addr 172.21.15.110:0/3939456119 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964ed9d640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9650102240 0x7f9650103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964e59c640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9650100690 0x7f96501047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 -- 172.21.15.110:0/3939456119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9650102240 msgr2=0x7f9650103340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9650102240 0x7f9650103340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 -- 172.21.15.110:0/3939456119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9650100690 msgr2=0x7f96501047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9650100690 0x7f96501047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 -- 172.21.15.110:0/3939456119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9650103bd0 con 0x7f9650101880 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964e59c640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9650100690 0x7f96501047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f964dd9b640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650101880 0x7f9650102e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f964400e9f0 tx=0x7f964400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f963f7fe640 1 -- 172.21.15.110:0/3939456119 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400cd80 con 0x7f9650101880 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96501a1ae0 con 0x7f9650101880 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f963f7fe640 1 -- 172.21.15.110:0/3939456119 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96440108e0 con 0x7f9650101880 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.606+0000 7f963f7fe640 1 -- 172.21.15.110:0/3939456119 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644018920 con 0x7f9650101880 2024-04-24T23:59:40.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.607+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96501a2030 con 0x7f9650101880 2024-04-24T23:59:40.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.607+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96501a2200 con 0x7f9650101880 2024-04-24T23:59:40.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.609+0000 7f963f7fe640 1 -- 172.21.15.110:0/3939456119 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9644010a90 con 0x7f9650101880 2024-04-24T23:59:40.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.610+0000 7f963f7fe640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f96380a2b90 0x7f96380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:40.610 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.610+0000 7f964e59c640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f96380a2b90 0x7f96380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:40.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.610+0000 7f963f7fe640 1 -- 172.21.15.110:0/3939456119 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f96440cf450 con 0x7f9650101880 2024-04-24T23:59:40.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.610+0000 7f964e59c640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f96380a2b90 0x7f96380a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9648000c00 tx=0x7f9648032c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:40.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.613+0000 7f963f7fe640 1 -- 172.21.15.110:0/3939456119 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f964409c850 con 0x7f9650101880 2024-04-24T23:59:40.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.836+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96501a24f0 con 0x7f9650101880 2024-04-24T23:59:40.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.837+0000 7f963f7fe640 1 -- 172.21.15.110:0/3939456119 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f964409c150 con 0x7f9650101880 2024-04-24T23:59:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:40.837 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T23:59:40.837 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 6 2024-04-24T23:59:40.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f96380a2b90 msgr2=0x7f96380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:40.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f96380a2b90 0x7f96380a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9648000c00 tx=0x7f9648032c40 comp rx=0 tx=0).stop 2024-04-24T23:59:40.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650101880 msgr2=0x7f9650102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:40.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650101880 0x7f9650102e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f964400e9f0 tx=0x7f964400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 shutdown_connections 2024-04-24T23:59:40.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f96380a2b90 0x7f96380a5050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9650102240 0x7f9650103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9650101880 0x7f9650102e00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 --2- 172.21.15.110:0/3939456119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9650100690 0x7f96501047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:40.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 >> 172.21.15.110:0/3939456119 conn(0x7f96500fbc40 msgr2=0x7f9650069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:40.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 shutdown_connections 2024-04-24T23:59:40.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:40.839+0000 7f9654f14640 1 -- 172.21.15.110:0/3939456119 wait complete. 2024-04-24T23:59:41.017 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 podman[63174]: 2024-04-24 23:59:40.581222526 +0000 UTC m=+0.402080892 container start 3ceaca1cfe29790e3189b1bb75d8da64a38939364c617dcdb8e39f10eddbbb14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T23:59:41.017 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 ceph-mds[63193]: -- 172.21.15.188:0/1154107650 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c7cdcfa40 con 0x564c7cda2800 2024-04-24T23:59:41.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f[63189]: starting mds.f at 2024-04-24T23:59:41.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 bash[63174]: 3ceaca1cfe29790e3189b1bb75d8da64a38939364c617dcdb8e39f10eddbbb14 2024-04-24T23:59:41.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 24 23:59:40 smithi188 systemd[1]: Started Ceph mds.f for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:41.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:41 smithi188 ceph-mon[33285]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:41.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:41.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:41.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:41.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:41.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:41.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:41 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:41.708 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:41 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3939456119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:41 smithi173 ceph-mon[32402]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:41 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:41.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:41 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3939456119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:41 smithi110 ceph-mon[29187]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:42.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:42.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:42.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:42.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:41 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:42.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:41 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3939456119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:42.724 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:42 smithi188 ceph-mon[33285]: Deploying daemon mds.c on smithi188 2024-04-24T23:59:42.724 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:42 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.188:6834/946560580,v1:172.21.15.188:6835/946560580] up:boot 2024-04-24T23:59:42.724 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:42 smithi188 ceph-mon[33285]: fsmap 6 up:standby 2024-04-24T23:59:42.724 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:42 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:59:42.850 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:42.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:42 smithi173 ceph-mon[32402]: Deploying daemon mds.c on smithi188 2024-04-24T23:59:42.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:42 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.188:6834/946560580,v1:172.21.15.188:6835/946560580] up:boot 2024-04-24T23:59:42.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:42 smithi173 ceph-mon[32402]: fsmap 6 up:standby 2024-04-24T23:59:42.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:42 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:59:42.876 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:42 smithi110 ceph-mon[29187]: Deploying daemon mds.c on smithi188 2024-04-24T23:59:42.876 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:42 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.188:6834/946560580,v1:172.21.15.188:6835/946560580] up:boot 2024-04-24T23:59:42.876 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:42 smithi110 ceph-mon[29187]: fsmap 6 up:standby 2024-04-24T23:59:42.876 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:42 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:59:42.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3129754682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d0102da0 msgr2=0x7fb5d01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3129754682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d0102da0 0x7fb5d01031c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009900 tx=0x7fb5c402a410 comp rx=0 tx=0).stop 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3129754682 shutdown_connections 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3129754682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d01001d0 0x7fb5d0109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3129754682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d00ff810 0x7fb5d00ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3129754682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d0102da0 0x7fb5d01031c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3129754682 >> 172.21.15.110:0/3129754682 conn(0x7fb5d00fb3b0 msgr2=0x7fb5d00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3129754682 shutdown_connections 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3129754682 wait complete. 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.960+0000 7fb5d6cd6640 1 Processor -- start 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d6cd6640 1 -- start start 2024-04-24T23:59:42.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d6cd6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d00ff810 0x7fb5d01024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d6cd6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d01001d0 0x7fb5d0100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d6cd6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0102da0 0x7fb5d0101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d6cd6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d010d320 con 0x7fb5d00ff810 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d6cd6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d010d1a0 con 0x7fb5d01001d0 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d6cd6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d010d4a0 con 0x7fb5d0102da0 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d00ff810 0x7fb5d01024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d00ff810 0x7fb5d01024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39886/0 (socket says 172.21.15.110:39886) 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 -- 172.21.15.110:0/3376628603 learned_addr learned my addr 172.21.15.110:0/3376628603 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d5273640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0102da0 0x7fb5d0101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5cffff640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d01001d0 0x7fb5d0100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 -- 172.21.15.110:0/3376628603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0102da0 msgr2=0x7fb5d0101020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0102da0 0x7fb5d0101020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:42.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 -- 172.21.15.110:0/3376628603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d01001d0 msgr2=0x7fb5d0100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d01001d0 0x7fb5d0100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 -- 172.21.15.110:0/3376628603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d01018b0 con 0x7fb5d00ff810 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5cffff640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d01001d0 0x7fb5d0100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d4a72640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d00ff810 0x7fb5d01024a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40314e0 tx=0x7fb5c4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5cdffb640 1 -- 172.21.15.110:0/3376628603 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c403f070 con 0x7fb5d00ff810 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5cdffb640 1 -- 172.21.15.110:0/3376628603 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5c4031c40 con 0x7fb5d00ff810 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d01a1a90 con 0x7fb5d00ff810 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.961+0000 7fb5cdffb640 1 -- 172.21.15.110:0/3376628603 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4035030 con 0x7fb5d00ff810 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.962+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d01a1fe0 con 0x7fb5d00ff810 2024-04-24T23:59:42.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.963+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d0108c60 con 0x7fb5d00ff810 2024-04-24T23:59:42.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.964+0000 7fb5cdffb640 1 -- 172.21.15.110:0/3376628603 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb5c4042070 con 0x7fb5d00ff810 2024-04-24T23:59:42.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.964+0000 7fb5cdffb640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb5c00a2b90 0x7fb5c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:42.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.965+0000 7fb5cffff640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb5c00a2b90 0x7fb5c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:42.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.965+0000 7fb5cdffb640 1 -- 172.21.15.110:0/3376628603 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb5c40ef440 con 0x7fb5d00ff810 2024-04-24T23:59:42.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.965+0000 7fb5cffff640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb5c00a2b90 0x7fb5c00a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0058c0 tx=0x7fb5bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:42.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:42.969+0000 7fb5cdffb640 1 -- 172.21.15.110:0/3376628603 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb5c40bc7e0 con 0x7fb5d00ff810 2024-04-24T23:59:43.017 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:42 smithi188 systemd[1]: Starting Ceph mds.c for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:43.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.179+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5d002d170 con 0x7fb5d00ff810 2024-04-24T23:59:43.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.180+0000 7fb5cdffb640 1 -- 172.21.15.110:0/3376628603 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fb5c40bc0e0 con 0x7fb5d00ff810 2024-04-24T23:59:43.181 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:43.181 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T23:59:43.182 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 7 2024-04-24T23:59:43.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb5c00a2b90 msgr2=0x7fb5c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:43.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb5c00a2b90 0x7fb5c00a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0058c0 tx=0x7fb5bc005850 comp rx=0 tx=0).stop 2024-04-24T23:59:43.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d00ff810 msgr2=0x7fb5d01024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:43.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d00ff810 0x7fb5d01024a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40314e0 tx=0x7fb5c4003c10 comp rx=0 tx=0).stop 2024-04-24T23:59:43.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 shutdown_connections 2024-04-24T23:59:43.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb5c00a2b90 0x7fb5c00a5050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:43.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0102da0 0x7fb5d0101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:43.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d01001d0 0x7fb5d0100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:43.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 --2- 172.21.15.110:0/3376628603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5d00ff810 0x7fb5d01024a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:43.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 >> 172.21.15.110:0/3376628603 conn(0x7fb5d00fb3b0 msgr2=0x7fb5d0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:43.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 shutdown_connections 2024-04-24T23:59:43.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:43.182+0000 7fb5d6cd6640 1 -- 172.21.15.110:0/3376628603 wait complete. 2024-04-24T23:59:43.426 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:43 smithi188 podman[63482]: 2024-04-24 23:59:43.040723031 +0000 UTC m=+0.018129536 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:43.426 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:43 smithi188 podman[63482]: 2024-04-24 23:59:43.165287382 +0000 UTC m=+0.142693872 container create 05ee9d634df9d94bda33935d5b8868554eb512110af4d6a6a419fc9b783ab920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T23:59:43.426 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:43 smithi188 podman[63482]: 2024-04-24 23:59:43.426712326 +0000 UTC m=+0.404118824 container init 05ee9d634df9d94bda33935d5b8868554eb512110af4d6a6a419fc9b783ab920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:59:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:43 smithi188 ceph-mon[33285]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:43 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3376628603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:43.768 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:43 smithi188 podman[63482]: 2024-04-24 23:59:43.429334681 +0000 UTC m=+0.406741170 container start 05ee9d634df9d94bda33935d5b8868554eb512110af4d6a6a419fc9b783ab920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-24T23:59:43.768 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:43 smithi188 ceph-mds[63501]: -- 172.21.15.188:0/3795251257 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640f9fcba40 con 0x5640f9f9f000 2024-04-24T23:59:43.768 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:43 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c[63497]: starting mds.c at 2024-04-24T23:59:43.768 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:43 smithi188 bash[63482]: 05ee9d634df9d94bda33935d5b8868554eb512110af4d6a6a419fc9b783ab920 2024-04-24T23:59:43.768 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 24 23:59:43 smithi188 systemd[1]: Started Ceph mds.c for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:43.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:43 smithi173 ceph-mon[32402]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:43.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:43 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3376628603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:43 smithi110 ceph-mon[29187]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:43 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3376628603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:44.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.188:6836/1170231083,v1:172.21.15.188:6837/1170231083] up:boot 2024-04-24T23:59:44.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: fsmap 7 up:standby 2024-04-24T23:59:44.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:59:44.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:44.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:44.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:44.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:44.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:44.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:44.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:44 smithi173 ceph-mon[32402]: Deploying daemon mds.h on smithi173 2024-04-24T23:59:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.188:6836/1170231083,v1:172.21.15.188:6837/1170231083] up:boot 2024-04-24T23:59:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: fsmap 7 up:standby 2024-04-24T23:59:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:59:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:44 smithi188 ceph-mon[33285]: Deploying daemon mds.h on smithi173 2024-04-24T23:59:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.188:6836/1170231083,v1:172.21.15.188:6837/1170231083] up:boot 2024-04-24T23:59:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: fsmap 7 up:standby 2024-04-24T23:59:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:59:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:45.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:45.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:45.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:45.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:45.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:45.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:44 smithi110 ceph-mon[29187]: Deploying daemon mds.h on smithi173 2024-04-24T23:59:45.195 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:45.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 -- 172.21.15.110:0/2352996628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c1006a0 msgr2=0x7f130c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:45.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 --2- 172.21.15.110:0/2352996628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c1006a0 0x7f130c100ac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1304009900 tx=0x7f130402a410 comp rx=0 tx=0).stop 2024-04-24T23:59:45.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 -- 172.21.15.110:0/2352996628 shutdown_connections 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 --2- 172.21.15.110:0/2352996628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c102250 0x7f130c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 --2- 172.21.15.110:0/2352996628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c101890 0x7f130c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 --2- 172.21.15.110:0/2352996628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c1006a0 0x7f130c100ac0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 -- 172.21.15.110:0/2352996628 >> 172.21.15.110:0/2352996628 conn(0x7f130c0fbc30 msgr2=0x7f130c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 -- 172.21.15.110:0/2352996628 shutdown_connections 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.303+0000 7f1310d70640 1 -- 172.21.15.110:0/2352996628 wait complete. 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f1310d70640 1 Processor -- start 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f1310d70640 1 -- start start 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f1310d70640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c1006a0 0x7f130c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f1310d70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c101890 0x7f130c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f1310d70640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c102250 0x7f130c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f1310d70640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f130c10c220 con 0x7f130c102250 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f1310d70640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f130c10c0a0 con 0x7f130c1006a0 2024-04-24T23:59:45.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f1310d70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f130c10c3a0 con 0x7f130c101890 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.304+0000 7f130ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c102250 0x7f130c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c102250 0x7f130c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39918/0 (socket says 172.21.15.110:39918) 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130ad9d640 1 -- 172.21.15.110:0/1958737449 learned_addr learned my addr 172.21.15.110:0/1958737449 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130a59c640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c1006a0 0x7f130c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f1309d9b640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c101890 0x7f130c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130ad9d640 1 -- 172.21.15.110:0/1958737449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c101890 msgr2=0x7f130c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130ad9d640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c101890 0x7f130c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130ad9d640 1 -- 172.21.15.110:0/1958737449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c1006a0 msgr2=0x7f130c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130ad9d640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c1006a0 0x7f130c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130ad9d640 1 -- 172.21.15.110:0/1958737449 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c19f5b0 con 0x7f130c102250 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130a59c640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c1006a0 0x7f130c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f130ad9d640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c102250 0x7f130c19ef40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009fd0 tx=0x7f12fc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f12fb7fe640 1 -- 172.21.15.110:0/1958737449 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc011830 con 0x7f130c102250 2024-04-24T23:59:45.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f12fb7fe640 1 -- 172.21.15.110:0/1958737449 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12fc016070 con 0x7f130c102250 2024-04-24T23:59:45.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c19f8a0 con 0x7f130c102250 2024-04-24T23:59:45.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f12fb7fe640 1 -- 172.21.15.110:0/1958737449 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc010de0 con 0x7f130c102250 2024-04-24T23:59:45.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.305+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c19fe70 con 0x7f130c102250 2024-04-24T23:59:45.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.306+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130c1a00a0 con 0x7f130c102250 2024-04-24T23:59:45.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.307+0000 7f12fb7fe640 1 -- 172.21.15.110:0/1958737449 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f12fc0119d0 con 0x7f130c102250 2024-04-24T23:59:45.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.308+0000 7f12fb7fe640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12f40a2b90 0x7f12f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:45.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.308+0000 7f130a59c640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12f40a2b90 0x7f12f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:45.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.308+0000 7f12fb7fe640 1 -- 172.21.15.110:0/1958737449 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f12fc0ce320 con 0x7f130c102250 2024-04-24T23:59:45.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.308+0000 7f130a59c640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12f40a2b90 0x7f12f40a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1304000c00 tx=0x7f1304031d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:45.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.312+0000 7f12fb7fe640 1 -- 172.21.15.110:0/1958737449 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12fc09b720 con 0x7f130c102250 2024-04-24T23:59:45.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.525+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f130c1a0310 con 0x7f130c102250 2024-04-24T23:59:45.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.525+0000 7f12fb7fe640 1 -- 172.21.15.110:0/1958737449 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f12fc09b020 con 0x7f130c102250 2024-04-24T23:59:45.526 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:45.526 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:59:45.526 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 8 2024-04-24T23:59:45.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12f40a2b90 msgr2=0x7f12f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12f40a2b90 0x7f12f40a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1304000c00 tx=0x7f1304031d10 comp rx=0 tx=0).stop 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c102250 msgr2=0x7f130c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c102250 0x7f130c19ef40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009fd0 tx=0x7f12fc00b8e0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 shutdown_connections 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12f40a2b90 0x7f12f40a5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f130c102250 0x7f130c19ef40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c101890 0x7f130c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 --2- 172.21.15.110:0/1958737449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c1006a0 0x7f130c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 >> 172.21.15.110:0/1958737449 conn(0x7f130c0fbc30 msgr2=0x7f130c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 shutdown_connections 2024-04-24T23:59:45.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:45.527+0000 7f1310d70640 1 -- 172.21.15.110:0/1958737449 wait complete. 2024-04-24T23:59:45.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:45 smithi173 ceph-mon[32402]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:45.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:45 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1958737449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:45.852 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:45 smithi173 systemd[1]: Starting Ceph mds.h for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:45 smithi188 ceph-mon[33285]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:45 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1958737449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:45 smithi110 ceph-mon[29187]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:45 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1958737449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:46.199 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:45 smithi173 podman[62543]: 2024-04-24 23:59:45.841427305 +0000 UTC m=+0.018430934 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:46.199 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:45 smithi173 podman[62543]: 2024-04-24 23:59:45.963809266 +0000 UTC m=+0.140812928 container create 1172453ed4452b36b17a03010626cb35194d13e81417dabaa36ed7ff4bc51329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:59:46.199 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:46 smithi173 podman[62543]: 2024-04-24 23:59:46.199725998 +0000 UTC m=+0.376729614 container init 1172453ed4452b36b17a03010626cb35194d13e81417dabaa36ed7ff4bc51329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-24T23:59:46.602 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:46 smithi173 podman[62543]: 2024-04-24 23:59:46.202264242 +0000 UTC m=+0.379267859 container start 1172453ed4452b36b17a03010626cb35194d13e81417dabaa36ed7ff4bc51329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:59:46.602 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:46 smithi173 ceph-mds[62563]: -- 172.21.15.173:0/94771461 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c35e3ba40 con 0x555c35e0e800 2024-04-24T23:59:46.602 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:46 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h[62559]: starting mds.h at 2024-04-24T23:59:46.602 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:46 smithi173 bash[62543]: 1172453ed4452b36b17a03010626cb35194d13e81417dabaa36ed7ff4bc51329 2024-04-24T23:59:46.602 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 24 23:59:46 smithi173 systemd[1]: Started Ceph mds.h for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:47.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:47.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:47.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:47.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: Deploying daemon mds.i on smithi188 2024-04-24T23:59:47.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:47.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] up:boot 2024-04-24T23:59:47.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: fsmap 8 up:standby 2024-04-24T23:59:47.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:59:47.540 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: Deploying daemon mds.i on smithi188 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:47.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] up:boot 2024-04-24T23:59:47.603 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: fsmap 8 up:standby 2024-04-24T23:59:47.603 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:59:47.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 -- 172.21.15.110:0/2742143230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684102250 msgr2=0x7f8684108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:47.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 --2- 172.21.15.110:0/2742143230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684102250 0x7f8684108ae0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8680009ca0 tx=0x7f868002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 -- 172.21.15.110:0/2742143230 shutdown_connections 2024-04-24T23:59:47.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 --2- 172.21.15.110:0/2742143230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684102250 0x7f8684108ae0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 --2- 172.21.15.110:0/2742143230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8684101890 0x7f8684101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 --2- 172.21.15.110:0/2742143230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86841006a0 0x7f8684100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 -- 172.21.15.110:0/2742143230 >> 172.21.15.110:0/2742143230 conn(0x7f86840fbc30 msgr2=0x7f86840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 -- 172.21.15.110:0/2742143230 shutdown_connections 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 -- 172.21.15.110:0/2742143230 wait complete. 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.649+0000 7f868c6da640 1 Processor -- start 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868c6da640 1 -- start start 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868c6da640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86841006a0 0x7f868406a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868c6da640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684101890 0x7f868419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868c6da640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8684102250 0x7f868419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868c6da640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f868410c1d0 con 0x7f8684101890 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868c6da640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f868410c050 con 0x7f86841006a0 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868c6da640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f868410c350 con 0x7f8684102250 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684101890 0x7f868419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:47.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684101890 0x7f868419c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40842/0 (socket says 172.21.15.110:40842) 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 -- 172.21.15.110:0/1116351875 learned_addr learned my addr 172.21.15.110:0/1116351875 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868ac77640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8684102250 0x7f868419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868a476640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86841006a0 0x7f868406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 -- 172.21.15.110:0/1116351875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8684102250 msgr2=0x7f868419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8684102250 0x7f868419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 -- 172.21.15.110:0/1116351875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86841006a0 msgr2=0x7f868406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86841006a0 0x7f868406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 -- 172.21.15.110:0/1116351875 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868419f510 con 0x7f8684101890 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868a476640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86841006a0 0x7f868406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f8689c75640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684101890 0x7f868419c610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f867400e9f0 tx=0x7f867400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f867b7fe640 1 -- 172.21.15.110:0/1116351875 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867400cd80 con 0x7f8684101890 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868419f800 con 0x7f8684101890 2024-04-24T23:59:47.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.650+0000 7f867b7fe640 1 -- 172.21.15.110:0/1116351875 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86740054d0 con 0x7f8684101890 2024-04-24T23:59:47.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.651+0000 7f867b7fe640 1 -- 172.21.15.110:0/1116351875 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674010490 con 0x7f8684101890 2024-04-24T23:59:47.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.651+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868419fd00 con 0x7f8684101890 2024-04-24T23:59:47.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.651+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8684107a40 con 0x7f8684101890 2024-04-24T23:59:47.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.653+0000 7f867b7fe640 1 -- 172.21.15.110:0/1116351875 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8674005000 con 0x7f8684101890 2024-04-24T23:59:47.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.653+0000 7f867b7fe640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f86700a2b90 0x7f86700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:47.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.653+0000 7f868a476640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f86700a2b90 0x7f86700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:47.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.654+0000 7f867b7fe640 1 -- 172.21.15.110:0/1116351875 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f86740cf1f0 con 0x7f8684101890 2024-04-24T23:59:47.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.654+0000 7f868a476640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f86700a2b90 0x7f86700a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f86841016f0 tx=0x7f867c00a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:47.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.657+0000 7f867b7fe640 1 -- 172.21.15.110:0/1116351875 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8674053740 con 0x7f8684101890 2024-04-24T23:59:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: Deploying daemon mds.i on smithi188 2024-04-24T23:59:47.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] up:boot 2024-04-24T23:59:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: fsmap 8 up:standby 2024-04-24T23:59:47.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:59:47.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.868+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f868402d170 con 0x7f8684101890 2024-04-24T23:59:47.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.869+0000 7f867b7fe640 1 -- 172.21.15.110:0/1116351875 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f86740052f0 con 0x7f8684101890 2024-04-24T23:59:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:47.869 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:59:47.870 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 9 2024-04-24T23:59:47.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f86700a2b90 msgr2=0x7f86700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:47.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f86700a2b90 0x7f86700a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f86841016f0 tx=0x7f867c00a5e0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684101890 msgr2=0x7f868419c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684101890 0x7f868419c610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f867400e9f0 tx=0x7f867400edf0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 shutdown_connections 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f86700a2b90 0x7f86700a5050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8684102250 0x7f868419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8684101890 0x7f868419c610 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 --2- 172.21.15.110:0/1116351875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86841006a0 0x7f868406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 >> 172.21.15.110:0/1116351875 conn(0x7f86840fbc30 msgr2=0x7f8684104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 shutdown_connections 2024-04-24T23:59:47.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:47.871+0000 7f868c6da640 1 -- 172.21.15.110:0/1116351875 wait complete. 2024-04-24T23:59:48.726 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:48 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1116351875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:48.730 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:48 smithi188 systemd[1]: Starting Ceph mds.i for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:48.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:48 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1116351875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:48.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:48 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1116351875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:49.016 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:48 smithi188 podman[63790]: 2024-04-24 23:59:48.62923805 +0000 UTC m=+0.018119791 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:49.016 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:48 smithi188 podman[63790]: 2024-04-24 23:59:48.755870071 +0000 UTC m=+0.144751799 container create e29352a6f3e51dc692d648818e2fc5081f11cacddad5c70c15c8d16244e04144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d2c45d9) 2024-04-24T23:59:49.016 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:49 smithi188 podman[63790]: 2024-04-24 23:59:49.01716354 +0000 UTC m=+0.406045275 container init e29352a6f3e51dc692d648818e2fc5081f11cacddad5c70c15c8d16244e04144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:59:49.267 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:49 smithi188 podman[63790]: 2024-04-24 23:59:49.019843563 +0000 UTC m=+0.408725297 container start e29352a6f3e51dc692d648818e2fc5081f11cacddad5c70c15c8d16244e04144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T23:59:49.268 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-mds[63809]: -- 172.21.15.188:0/90488363 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569b796da40 con 0x5569b7940800 2024-04-24T23:59:49.268 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i[63805]: starting mds.i at 2024-04-24T23:59:49.268 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:49 smithi188 bash[63790]: e29352a6f3e51dc692d648818e2fc5081f11cacddad5c70c15c8d16244e04144 2024-04-24T23:59:49.268 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 24 23:59:49 smithi188 systemd[1]: Started Ceph mds.i for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-mon[33285]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:49.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:49.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:49.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:49 smithi110 ceph-mon[29187]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:49.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:49.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:49.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:49 smithi173 ceph-mon[32402]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:49.883 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 -- 172.21.15.110:0/986740136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8100680 msgr2=0x7f05b8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 --2- 172.21.15.110:0/986740136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8100680 0x7f05b8100aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009900 tx=0x7f05b402a410 comp rx=0 tx=0).stop 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 -- 172.21.15.110:0/986740136 shutdown_connections 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 --2- 172.21.15.110:0/986740136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8102230 0x7f05b8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 --2- 172.21.15.110:0/986740136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b8101870 0x7f05b8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 --2- 172.21.15.110:0/986740136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8100680 0x7f05b8100aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 -- 172.21.15.110:0/986740136 >> 172.21.15.110:0/986740136 conn(0x7f05b80fbbf0 msgr2=0x7f05b80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 -- 172.21.15.110:0/986740136 shutdown_connections 2024-04-24T23:59:49.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.986+0000 7f05c05e2640 1 -- 172.21.15.110:0/986740136 wait complete. 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05c05e2640 1 Processor -- start 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05c05e2640 1 -- start start 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05c05e2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8100680 0x7f05b8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05c05e2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8101870 0x7f05b8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05c05e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b8102230 0x7f05b819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05c05e2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05b810d330 con 0x7f05b8101870 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05c05e2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05b810d1b0 con 0x7f05b8100680 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05c05e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f05b810d4b0 con 0x7f05b8102230 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05bdb7d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8101870 0x7f05b8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05bdb7d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8101870 0x7f05b8197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40854/0 (socket says 172.21.15.110:40854) 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05bdb7d640 1 -- 172.21.15.110:0/3354759698 learned_addr learned my addr 172.21.15.110:0/3354759698 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05beb7f640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b8102230 0x7f05b819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.987+0000 7f05be37e640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8100680 0x7f05b8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:49.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05bdb7d640 1 -- 172.21.15.110:0/3354759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b8102230 msgr2=0x7f05b819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05bdb7d640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b8102230 0x7f05b819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05bdb7d640 1 -- 172.21.15.110:0/3354759698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8100680 msgr2=0x7f05b8197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05bdb7d640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8100680 0x7f05b8197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05bdb7d640 1 -- 172.21.15.110:0/3354759698 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b819f660 con 0x7f05b8101870 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05be37e640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8100680 0x7f05b8197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05bdb7d640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8101870 0x7f05b8197f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05a800d770 tx=0x7f05a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05af7fe640 1 -- 172.21.15.110:0/3354759698 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8017070 con 0x7f05b8101870 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05af7fe640 1 -- 172.21.15.110:0/3354759698 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05a80054d0 con 0x7f05b8101870 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b819f950 con 0x7f05b8101870 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05af7fe640 1 -- 172.21.15.110:0/3354759698 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8004cf0 con 0x7f05b8101870 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.988+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b819fe50 con 0x7f05b8101870 2024-04-24T23:59:49.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.989+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b8107a40 con 0x7f05b8101870 2024-04-24T23:59:49.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.990+0000 7f05af7fe640 1 -- 172.21.15.110:0/3354759698 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f05a8004e90 con 0x7f05b8101870 2024-04-24T23:59:49.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.991+0000 7f05af7fe640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05a40a2b90 0x7f05a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:49.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.991+0000 7f05be37e640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05a40a2b90 0x7f05a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:49.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.991+0000 7f05af7fe640 1 -- 172.21.15.110:0/3354759698 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f05a80ce610 con 0x7f05b8101870 2024-04-24T23:59:49.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.991+0000 7f05be37e640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05a40a2b90 0x7f05a40a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006010 tx=0x7f05b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:49.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:49.995+0000 7f05af7fe640 1 -- 172.21.15.110:0/3354759698 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f05a809ba10 con 0x7f05b8101870 2024-04-24T23:59:50.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.215+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05b802d170 con 0x7f05b8101870 2024-04-24T23:59:50.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.216+0000 7f05af7fe640 1 -- 172.21.15.110:0/3354759698 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f05a809b310 con 0x7f05b8101870 2024-04-24T23:59:50.216 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:50.216 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:59:50.216 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 10 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05a40a2b90 msgr2=0x7f05a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05a40a2b90 0x7f05a40a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006010 tx=0x7f05b4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8101870 msgr2=0x7f05b8197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8101870 0x7f05b8197f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05a800d770 tx=0x7f05a800dc30 comp rx=0 tx=0).stop 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 shutdown_connections 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05a40a2b90 0x7f05a40a5050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b8102230 0x7f05b819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b8101870 0x7f05b8197f80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:50.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 --2- 172.21.15.110:0/3354759698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b8100680 0x7f05b8197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:50.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 >> 172.21.15.110:0/3354759698 conn(0x7f05b80fbbf0 msgr2=0x7f05b8104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:50.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 shutdown_connections 2024-04-24T23:59:50.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:50.218+0000 7f05c05e2640 1 -- 172.21.15.110:0/3354759698 wait complete. 2024-04-24T23:59:50.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:50 smithi188 ceph-mon[33285]: Deploying daemon mds.g on smithi110 2024-04-24T23:59:50.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:50 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] up:boot 2024-04-24T23:59:50.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:50 smithi188 ceph-mon[33285]: fsmap 9 up:standby 2024-04-24T23:59:50.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:50 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:59:50.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:50 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3354759698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:50.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:50 smithi110 ceph-mon[29187]: Deploying daemon mds.g on smithi110 2024-04-24T23:59:50.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:50 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] up:boot 2024-04-24T23:59:50.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:50 smithi110 ceph-mon[29187]: fsmap 9 up:standby 2024-04-24T23:59:50.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:50 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:59:50.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:50 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3354759698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:50 smithi173 ceph-mon[32402]: Deploying daemon mds.g on smithi110 2024-04-24T23:59:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:50 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] up:boot 2024-04-24T23:59:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:50 smithi173 ceph-mon[32402]: fsmap 9 up:standby 2024-04-24T23:59:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:50 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:59:50.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:50 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3354759698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:51.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:51 smithi110 ceph-mon[29187]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:51 smithi188 ceph-mon[33285]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:51 smithi173 ceph-mon[32402]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:52.230 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:52.252 INFO:journalctl@ceph.mds.g.smithi110.stdout:Apr 24 23:59:51 smithi110 systemd[1]: Started Ceph mds.g for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:52.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.333+0000 7f7713522640 1 -- 172.21.15.110:0/3311472244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c101870 msgr2=0x7f770c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.333+0000 7f7713522640 1 --2- 172.21.15.110:0/3311472244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c101870 0x7f770c101cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009900 tx=0x7f76fc02a410 comp rx=0 tx=0).stop 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.333+0000 7f7713522640 1 -- 172.21.15.110:0/3311472244 shutdown_connections 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.333+0000 7f7713522640 1 --2- 172.21.15.110:0/3311472244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f770c102230 0x7f770c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.333+0000 7f7713522640 1 --2- 172.21.15.110:0/3311472244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c101870 0x7f770c101cf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.333+0000 7f7713522640 1 --2- 172.21.15.110:0/3311472244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f770c100680 0x7f770c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.333+0000 7f7713522640 1 -- 172.21.15.110:0/3311472244 >> 172.21.15.110:0/3311472244 conn(0x7f770c0fbbf0 msgr2=0x7f770c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 -- 172.21.15.110:0/3311472244 shutdown_connections 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 -- 172.21.15.110:0/3311472244 wait complete. 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 Processor -- start 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 -- start start 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f770c100680 0x7f770c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f770c101870 0x7f770c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:52.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c102230 0x7f770c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f770c10c1b0 con 0x7f770c102230 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f770c10c030 con 0x7f770c100680 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7713522640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f770c10c330 con 0x7f770c101870 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7711abf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c102230 0x7f770c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7711abf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c102230 0x7f770c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40882/0 (socket says 172.21.15.110:40882) 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7711abf640 1 -- 172.21.15.110:0/2601565211 learned_addr learned my addr 172.21.15.110:0/2601565211 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f7710abd640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f770c101870 0x7f770c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.334+0000 7f77112be640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f770c100680 0x7f770c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f7711abf640 1 -- 172.21.15.110:0/2601565211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f770c101870 msgr2=0x7f770c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f7711abf640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f770c101870 0x7f770c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f7711abf640 1 -- 172.21.15.110:0/2601565211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f770c100680 msgr2=0x7f770c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f7711abf640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f770c100680 0x7f770c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f7711abf640 1 -- 172.21.15.110:0/2601565211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c19f660 con 0x7f770c102230 2024-04-24T23:59:52.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f7711abf640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c102230 0x7f770c19ed50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f770400eaa0 tx=0x7f770400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:52.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f77027fc640 1 -- 172.21.15.110:0/2601565211 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400cdb0 con 0x7f770c102230 2024-04-24T23:59:52.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c19f8f0 con 0x7f770c102230 2024-04-24T23:59:52.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f77027fc640 1 -- 172.21.15.110:0/2601565211 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7704005020 con 0x7f770c102230 2024-04-24T23:59:52.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f77027fc640 1 -- 172.21.15.110:0/2601565211 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040116d0 con 0x7f770c102230 2024-04-24T23:59:52.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.335+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c19fe10 con 0x7f770c102230 2024-04-24T23:59:52.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.336+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c068560 con 0x7f770c102230 2024-04-24T23:59:52.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.337+0000 7f77027fc640 1 -- 172.21.15.110:0/2601565211 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f77040051d0 con 0x7f770c102230 2024-04-24T23:59:52.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.337+0000 7f77027fc640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76f80a2b90 0x7f76f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:52.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.338+0000 7f77112be640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76f80a2b90 0x7f76f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:52.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.338+0000 7f77027fc640 1 -- 172.21.15.110:0/2601565211 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f77040cecf0 con 0x7f770c102230 2024-04-24T23:59:52.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.338+0000 7f77112be640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76f80a2b90 0x7f76f80a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77080058c0 tx=0x7f770800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:52.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.342+0000 7f77027fc640 1 -- 172.21.15.110:0/2601565211 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77040d3070 con 0x7f770c102230 2024-04-24T23:59:52.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.614+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f770c02d170 con 0x7f770c102230 2024-04-24T23:59:52.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.615+0000 7f77027fc640 1 -- 172.21.15.110:0/2601565211 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f7704016030 con 0x7f770c102230 2024-04-24T23:59:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:52.615 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:59:52.616 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 11 2024-04-24T23:59:52.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76f80a2b90 msgr2=0x7f76f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:52.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76f80a2b90 0x7f76f80a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77080058c0 tx=0x7f770800a2d0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c102230 msgr2=0x7f770c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:52.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c102230 0x7f770c19ed50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f770400eaa0 tx=0x7f770400ef60 comp rx=0 tx=0).stop 2024-04-24T23:59:52.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 shutdown_connections 2024-04-24T23:59:52.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76f80a2b90 0x7f76f80a5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f770c102230 0x7f770c19ed50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f770c101870 0x7f770c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 --2- 172.21.15.110:0/2601565211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f770c100680 0x7f770c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:52.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 >> 172.21.15.110:0/2601565211 conn(0x7f770c0fbbf0 msgr2=0x7f770c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:52.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 shutdown_connections 2024-04-24T23:59:52.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:52.617+0000 7f7713522640 1 -- 172.21.15.110:0/2601565211 wait complete. 2024-04-24T23:59:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: Deploying daemon mds.a on smithi110 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.110:6838/717154296,v1:172.21.15.110:6839/717154296] up:boot 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: fsmap 10 up:standby 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:53 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2601565211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: Deploying daemon mds.a on smithi110 2024-04-24T23:59:53.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.110:6838/717154296,v1:172.21.15.110:6839/717154296] up:boot 2024-04-24T23:59:53.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: fsmap 10 up:standby 2024-04-24T23:59:53.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:59:53.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:53 smithi110 ceph-mon[29187]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:53.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:53 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2601565211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: Deploying daemon mds.a on smithi110 2024-04-24T23:59:53.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.110:6838/717154296,v1:172.21.15.110:6839/717154296] up:boot 2024-04-24T23:59:53.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: fsmap 10 up:standby 2024-04-24T23:59:53.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:59:53.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:53.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:53 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2601565211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:54.536 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 systemd[1]: Starting Ceph mds.a for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:54.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.730+0000 7fd922ee1640 1 -- 172.21.15.110:0/2378606560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 msgr2=0x7fd91c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.730+0000 7fd922ee1640 1 --2- 172.21.15.110:0/2378606560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 0x7fd91c1098b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd914009ca0 tx=0x7fd91402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 -- 172.21.15.110:0/2378606560 shutdown_connections 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 --2- 172.21.15.110:0/2378606560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 0x7fd91c1098b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 --2- 172.21.15.110:0/2378606560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd91c104b30 0x7fd91c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 --2- 172.21.15.110:0/2378606560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd91c102200 0x7fd91c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 -- 172.21.15.110:0/2378606560 >> 172.21.15.110:0/2378606560 conn(0x7fd91c0fbbf0 msgr2=0x7fd91c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 -- 172.21.15.110:0/2378606560 shutdown_connections 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 -- 172.21.15.110:0/2378606560 wait complete. 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 Processor -- start 2024-04-24T23:59:54.733 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 -- start start 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd91c102200 0x7fd91c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd91c104b30 0x7fd91c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 0x7fd91c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d340 con 0x7fd91c107460 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d1c0 con 0x7fd91c104b30 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.731+0000 7fd922ee1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d4c0 con 0x7fd91c102200 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 0x7fd91c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 0x7fd91c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40896/0 (socket says 172.21.15.110:40896) 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 -- 172.21.15.110:0/1835631185 learned_addr learned my addr 172.21.15.110:0/1835631185 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd920c7d640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd91c102200 0x7fd91c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd913fff640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd91c104b30 0x7fd91c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 -- 172.21.15.110:0/1835631185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd91c102200 msgr2=0x7fd91c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd91c102200 0x7fd91c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 -- 172.21.15.110:0/1835631185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd91c104b30 msgr2=0x7fd91c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd91c104b30 0x7fd91c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 -- 172.21.15.110:0/1835631185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c19f610 con 0x7fd91c107460 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd920c7d640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd91c102200 0x7fd91c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd913fff640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd91c104b30 0x7fd91c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd92147e640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 0x7fd91c19efa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd914009dc0 tx=0x7fd914003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd911ffb640 1 -- 172.21.15.110:0/1835631185 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914032ec0 con 0x7fd91c107460 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c19f8a0 con 0x7fd91c107460 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd911ffb640 1 -- 172.21.15.110:0/1835631185 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9140044f0 con 0x7fd91c107460 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd911ffb640 1 -- 172.21.15.110:0/1835631185 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914030d40 con 0x7fd91c107460 2024-04-24T23:59:54.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.732+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c19fe40 con 0x7fd91c107460 2024-04-24T23:59:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.734+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91c0fd6e0 con 0x7fd91c107460 2024-04-24T23:59:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.734+0000 7fd911ffb640 1 -- 172.21.15.110:0/1835631185 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd914004040 con 0x7fd91c107460 2024-04-24T23:59:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.735+0000 7fd911ffb640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd9040a2d30 0x7fd9040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.735+0000 7fd920c7d640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd9040a2d30 0x7fd9040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.735+0000 7fd911ffb640 1 -- 172.21.15.110:0/1835631185 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd9140f0340 con 0x7fd91c107460 2024-04-24T23:59:54.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.736+0000 7fd920c7d640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd9040a2d30 0x7fd9040a51f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd9180058c0 tx=0x7fd918005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:54.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.740+0000 7fd911ffb640 1 -- 172.21.15.110:0/1835631185 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd9140bd6a0 con 0x7fd91c107460 2024-04-24T23:59:54.865 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 podman[70640]: 2024-04-24 23:59:54.50037349 +0000 UTC m=+0.020591616 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:54.865 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 podman[70640]: 2024-04-24 23:59:54.612548337 +0000 UTC m=+0.132766445 container create bf534df8e285c50c84ceca4568650ac2129d457bdd77de38cca88e4aa900571a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T23:59:54.866 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 podman[70640]: 2024-04-24 23:59:54.865445071 +0000 UTC m=+0.385663177 container init bf534df8e285c50c84ceca4568650ac2129d457bdd77de38cca88e4aa900571a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T23:59:54.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.961+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd91c062340 con 0x7fd91c107460 2024-04-24T23:59:54.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.962+0000 7fd911ffb640 1 -- 172.21.15.110:0/1835631185 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7fd9140bcfa0 con 0x7fd91c107460 2024-04-24T23:59:54.962 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:54.962 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:59:54.962 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 11 2024-04-24T23:59:54.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd9040a2d30 msgr2=0x7fd9040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:54.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd9040a2d30 0x7fd9040a51f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd9180058c0 tx=0x7fd918005850 comp rx=0 tx=0).stop 2024-04-24T23:59:54.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 msgr2=0x7fd91c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:54.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 0x7fd91c19efa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd914009dc0 tx=0x7fd914003ba0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 shutdown_connections 2024-04-24T23:59:54.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd9040a2d30 0x7fd9040a51f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd91c107460 0x7fd91c19efa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd91c104b30 0x7fd91c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 --2- 172.21.15.110:0/1835631185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd91c102200 0x7fd91c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:54.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 >> 172.21.15.110:0/1835631185 conn(0x7fd91c0fbbf0 msgr2=0x7fd91c102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:54.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 shutdown_connections 2024-04-24T23:59:54.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:54.964+0000 7fd922ee1640 1 -- 172.21.15.110:0/1835631185 wait complete. 2024-04-24T23:59:55.286 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 podman[70640]: 2024-04-24 23:59:54.868424284 +0000 UTC m=+0.388642390 container start bf534df8e285c50c84ceca4568650ac2129d457bdd77de38cca88e4aa900571a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T23:59:55.286 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 ceph-mds[70698]: -- 172.21.15.110:0/1342037553 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55feba9b1a40 con 0x55feba985000 2024-04-24T23:59:55.286 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a[70694]: starting mds.a at 2024-04-24T23:59:55.286 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 bash[70640]: bf534df8e285c50c84ceca4568650ac2129d457bdd77de38cca88e4aa900571a 2024-04-24T23:59:55.286 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 24 23:59:54 smithi110 systemd[1]: Started Ceph mds.a for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:55 smithi173 ceph-mon[32402]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:55 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1835631185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:55.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:55.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:55 smithi188 ceph-mon[33285]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:55 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1835631185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:55 smithi110 ceph-mon[29187]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:55 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1835631185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:59:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:59:56.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:59:56.848 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:56 smithi173 ceph-mon[32402]: Deploying daemon mds.b on smithi173 2024-04-24T23:59:56.849 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:56 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.110:6840/2788119142,v1:172.21.15.110:6841/2788119142] up:boot 2024-04-24T23:59:56.849 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:56 smithi173 ceph-mon[32402]: fsmap 11 up:standby 2024-04-24T23:59:56.849 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:59:56.975 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:57.001 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:56 smithi110 ceph-mon[29187]: Deploying daemon mds.b on smithi173 2024-04-24T23:59:57.001 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:56 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.110:6840/2788119142,v1:172.21.15.110:6841/2788119142] up:boot 2024-04-24T23:59:57.001 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:56 smithi110 ceph-mon[29187]: fsmap 11 up:standby 2024-04-24T23:59:57.001 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:59:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:56 smithi188 ceph-mon[33285]: Deploying daemon mds.b on smithi173 2024-04-24T23:59:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:56 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.110:6840/2788119142,v1:172.21.15.110:6841/2788119142] up:boot 2024-04-24T23:59:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:56 smithi188 ceph-mon[33285]: fsmap 11 up:standby 2024-04-24T23:59:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:59:57.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 -- 172.21.15.110:0/3083917005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524102240 msgr2=0x7fa524108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:57.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/3083917005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524102240 0x7fa524108ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009ca0 tx=0x7fa51c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 -- 172.21.15.110:0/3083917005 shutdown_connections 2024-04-24T23:59:57.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/3083917005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524102240 0x7fa524108ad0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/3083917005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa524101880 0x7fa524101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/3083917005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa524100690 0x7fa524100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 -- 172.21.15.110:0/3083917005 >> 172.21.15.110:0/3083917005 conn(0x7fa5240fbc20 msgr2=0x7fa5240fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:57.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 -- 172.21.15.110:0/3083917005 shutdown_connections 2024-04-24T23:59:57.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 -- 172.21.15.110:0/3083917005 wait complete. 2024-04-24T23:59:57.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 Processor -- start 2024-04-24T23:59:57.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.087+0000 7fa52a7f9640 1 -- start start 2024-04-24T23:59:57.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa52a7f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa524100690 0x7fa524197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:57.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa52a7f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524101880 0x7fa524197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:57.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa52a7f9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa524102240 0x7fa52419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:57.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa52a7f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa52410d2e0 con 0x7fa524101880 2024-04-24T23:59:57.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa52a7f9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa52410d160 con 0x7fa524102240 2024-04-24T23:59:57.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa52a7f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa52410d460 con 0x7fa524100690 2024-04-24T23:59:57.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524101880 0x7fa524197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524101880 0x7fa524197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58082/0 (socket says 172.21.15.110:58082) 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 -- 172.21.15.110:0/1448598534 learned_addr learned my addr 172.21.15.110:0/1448598534 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa528d96640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa524102240 0x7fa52419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa523fff640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa524100690 0x7fa524197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 -- 172.21.15.110:0/1448598534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa524100690 msgr2=0x7fa524197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa524100690 0x7fa524197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 -- 172.21.15.110:0/1448598534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa524102240 msgr2=0x7fa52419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa524102240 0x7fa52419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 -- 172.21.15.110:0/1448598534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52419f590 con 0x7fa524101880 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa528d96640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa524102240 0x7fa52419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5237fe640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524101880 0x7fa524197eb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa51800e9f0 tx=0x7fa51800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:57.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5217fa640 1 -- 172.21.15.110:0/1448598534 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51800cd80 con 0x7fa524101880 2024-04-24T23:59:57.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52419f880 con 0x7fa524101880 2024-04-24T23:59:57.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5217fa640 1 -- 172.21.15.110:0/1448598534 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5180108e0 con 0x7fa524101880 2024-04-24T23:59:57.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa5217fa640 1 -- 172.21.15.110:0/1448598534 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518018920 con 0x7fa524101880 2024-04-24T23:59:57.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.088+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52419fdd0 con 0x7fa524101880 2024-04-24T23:59:57.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.090+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5241079b0 con 0x7fa524101880 2024-04-24T23:59:57.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.091+0000 7fa5217fa640 1 -- 172.21.15.110:0/1448598534 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa518010a90 con 0x7fa524101880 2024-04-24T23:59:57.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.091+0000 7fa5217fa640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa50c0a2b90 0x7fa50c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:57.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.092+0000 7fa523fff640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa50c0a2b90 0x7fa50c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:57.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.092+0000 7fa5217fa640 1 -- 172.21.15.110:0/1448598534 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa5180cee70 con 0x7fa524101880 2024-04-24T23:59:57.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.092+0000 7fa523fff640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa50c0a2b90 0x7fa50c0a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5241016e0 tx=0x7fa51400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:57.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.096+0000 7fa5217fa640 1 -- 172.21.15.110:0/1448598534 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa51809bc20 con 0x7fa524101880 2024-04-24T23:59:57.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.308+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa52402d170 con 0x7fa524101880 2024-04-24T23:59:57.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.308+0000 7fa5217fa640 1 -- 172.21.15.110:0/1448598534 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7fa518086020 con 0x7fa524101880 2024-04-24T23:59:57.308 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:57.309 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:59:57.309 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 12 2024-04-24T23:59:57.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa50c0a2b90 msgr2=0x7fa50c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa50c0a2b90 0x7fa50c0a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5241016e0 tx=0x7fa51400b390 comp rx=0 tx=0).stop 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524101880 msgr2=0x7fa524197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524101880 0x7fa524197eb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa51800e9f0 tx=0x7fa51800eeb0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 shutdown_connections 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa50c0a2b90 0x7fa50c0a5050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa524102240 0x7fa52419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa524101880 0x7fa524197eb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 --2- 172.21.15.110:0/1448598534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa524100690 0x7fa524197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 >> 172.21.15.110:0/1448598534 conn(0x7fa5240fbc20 msgr2=0x7fa524104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 shutdown_connections 2024-04-24T23:59:57.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:57.310+0000 7fa52a7f9640 1 -- 172.21.15.110:0/1448598534 wait complete. 2024-04-24T23:59:57.543 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 systemd[1]: Starting Ceph mds.b for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-24T23:59:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:57 smithi110 ceph-mon[29187]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:57 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1448598534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:57.813 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:57 smithi173 ceph-mon[32402]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:57.813 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:57 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1448598534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:57.813 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 podman[62852]: 2024-04-24 23:59:57.543366243 +0000 UTC m=+0.114552250 container create 29446a726e8df71e64420a032bb72629d9dc7d56fd875b4b5568dad25abba700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:59:57.813 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 podman[62852]: 2024-04-24 23:59:57.447055677 +0000 UTC m=+0.018241676 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:59:57.814 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 podman[62852]: 2024-04-24 23:59:57.788473465 +0000 UTC m=+0.359659431 container init 29446a726e8df71e64420a032bb72629d9dc7d56fd875b4b5568dad25abba700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-24T23:59:57.814 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 podman[62852]: 2024-04-24 23:59:57.791163244 +0000 UTC m=+0.362349220 container start 29446a726e8df71e64420a032bb72629d9dc7d56fd875b4b5568dad25abba700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T23:59:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:57 smithi188 ceph-mon[33285]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:57 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1448598534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:59:58.102 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 ceph-mds[62871]: -- 172.21.15.173:0/904213844 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56189796da40 con 0x561897940800 2024-04-24T23:59:58.102 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: starting mds.b at 2024-04-24T23:59:58.102 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 bash[62852]: 29446a726e8df71e64420a032bb72629d9dc7d56fd875b4b5568dad25abba700 2024-04-24T23:59:58.102 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 24 23:59:57 smithi173 systemd[1]: Started Ceph mds.b for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] up:boot 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: fsmap 12 up:standby 2024-04-24T23:59:59.195 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 24 23:59:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:59:59.201 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.201 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:59.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:59.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] up:boot 2024-04-24T23:59:59.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: fsmap 12 up:standby 2024-04-24T23:59:59.202 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 24 23:59:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:59:59.323 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:59:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-24T23:59:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:59:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:59:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] up:boot 2024-04-24T23:59:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: fsmap 12 up:standby 2024-04-24T23:59:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 24 23:59:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:59:59.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.424+0000 7fee8cf14640 1 -- 172.21.15.110:0/3253677772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88101870 msgr2=0x7fee88101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:59.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.424+0000 7fee8cf14640 1 --2- 172.21.15.110:0/3253677772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88101870 0x7fee88101cf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009900 tx=0x7fee7c02a410 comp rx=0 tx=0).stop 2024-04-24T23:59:59.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 -- 172.21.15.110:0/3253677772 shutdown_connections 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 --2- 172.21.15.110:0/3253677772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee88102230 0x7fee88108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 --2- 172.21.15.110:0/3253677772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88101870 0x7fee88101cf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 --2- 172.21.15.110:0/3253677772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee88100680 0x7fee88100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 -- 172.21.15.110:0/3253677772 >> 172.21.15.110:0/3253677772 conn(0x7fee880fbbf0 msgr2=0x7fee880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 -- 172.21.15.110:0/3253677772 shutdown_connections 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 -- 172.21.15.110:0/3253677772 wait complete. 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 Processor -- start 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 -- start start 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.425+0000 7fee8cf14640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee88100680 0x7fee88104800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8cf14640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee88101870 0x7fee88102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8cf14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88102230 0x7fee88103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:59.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8cf14640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee8810c1b0 con 0x7fee88102230 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8cf14640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fee8810c030 con 0x7fee88101870 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8cf14640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fee8810c330 con 0x7fee88100680 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee86d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88102230 0x7fee88103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee88100680 0x7fee88104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee86d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88102230 0x7fee88103380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58100/0 (socket says 172.21.15.110:58100) 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee86d9d640 1 -- 172.21.15.110:0/4073342749 learned_addr learned my addr 172.21.15.110:0/4073342749 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee85d9b640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee88101870 0x7fee88102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8659c640 1 -- 172.21.15.110:0/4073342749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee88101870 msgr2=0x7fee88102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8659c640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee88101870 0x7fee88102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8659c640 1 -- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88102230 msgr2=0x7fee88103380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8659c640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88102230 0x7fee88103380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8659c640 1 -- 172.21.15.110:0/4073342749 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee88103c10 con 0x7fee88100680 2024-04-24T23:59:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee86d9d640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88102230 0x7fee88103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:59:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8659c640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee88100680 0x7fee88104800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e9c0 tx=0x7fee7800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee777fe640 1 -- 172.21.15.110:0/4073342749 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7800ccc0 con 0x7fee88100680 2024-04-24T23:59:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee777fe640 1 -- 172.21.15.110:0/4073342749 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee780035f0 con 0x7fee88100680 2024-04-24T23:59:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee881a1b00 con 0x7fee88100680 2024-04-24T23:59:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee777fe640 1 -- 172.21.15.110:0/4073342749 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7800ce60 con 0x7fee88100680 2024-04-24T23:59:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.426+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee881a2020 con 0x7fee88100680 2024-04-24T23:59:59.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.427+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee88068560 con 0x7fee88100680 2024-04-24T23:59:59.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.429+0000 7fee777fe640 1 -- 172.21.15.110:0/4073342749 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fee780037a0 con 0x7fee88100680 2024-04-24T23:59:59.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.429+0000 7fee777fe640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fee700a2c10 0x7fee700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:59.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.429+0000 7fee85d9b640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fee700a2c10 0x7fee700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:59.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.429+0000 7fee777fe640 1 -- 172.21.15.110:0/4073342749 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fee780ceff0 con 0x7fee88100680 2024-04-24T23:59:59.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.430+0000 7fee85d9b640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fee700a2c10 0x7fee700a50d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fee7c006010 tx=0x7fee7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:59.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.435+0000 7fee777fe640 1 -- 172.21.15.110:0/4073342749 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee7809c3b0 con 0x7fee88100680 2024-04-24T23:59:59.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.668+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee881a2420 con 0x7fee88100680 2024-04-24T23:59:59.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.669+0000 7fee777fe640 1 -- 172.21.15.110:0/4073342749 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7fee7809bcb0 con 0x7fee88100680 2024-04-24T23:59:59.669 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-24T23:59:59.670 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-24T23:59:59.670 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-24T23:59:59.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fee700a2c10 msgr2=0x7fee700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:59.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fee700a2c10 0x7fee700a50d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fee7c006010 tx=0x7fee7c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee88100680 msgr2=0x7fee88104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:59.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee88100680 0x7fee88104800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e9c0 tx=0x7fee7800ee80 comp rx=0 tx=0).stop 2024-04-24T23:59:59.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 shutdown_connections 2024-04-24T23:59:59.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fee700a2c10 0x7fee700a50d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee88102230 0x7fee88103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee88101870 0x7fee88102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 --2- 172.21.15.110:0/4073342749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee88100680 0x7fee88104800 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 >> 172.21.15.110:0/4073342749 conn(0x7fee880fbbf0 msgr2=0x7fee880688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:59.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 shutdown_connections 2024-04-24T23:59:59.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.671+0000 7fee8cf14640 1 -- 172.21.15.110:0/4073342749 wait complete. 2024-04-24T23:59:59.683 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:59:59.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 -- 172.21.15.110:0/1792212255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054107460 msgr2=0x7f10541098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:59.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 --2- 172.21.15.110:0/1792212255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054107460 0x7f10541098b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f104c009ca0 tx=0x7f104c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 -- 172.21.15.110:0/1792212255 shutdown_connections 2024-04-24T23:59:59.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 --2- 172.21.15.110:0/1792212255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054107460 0x7f10541098b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 --2- 172.21.15.110:0/1792212255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1054104b30 0x7f1054106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 --2- 172.21.15.110:0/1792212255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1054102200 0x7f10541045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 -- 172.21.15.110:0/1792212255 >> 172.21.15.110:0/1792212255 conn(0x7f10540fbbf0 msgr2=0x7f10540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:59:59.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 -- 172.21.15.110:0/1792212255 shutdown_connections 2024-04-24T23:59:59.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.789+0000 7f105c2bd640 1 -- 172.21.15.110:0/1792212255 wait complete. 2024-04-24T23:59:59.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105c2bd640 1 Processor -- start 2024-04-24T23:59:59.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105c2bd640 1 -- start start 2024-04-24T23:59:59.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105c2bd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1054102200 0x7f1054197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:59.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105c2bd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054104b30 0x7f1054197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:59.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105c2bd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1054107460 0x7f105419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:59.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105c2bd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f105410d2d0 con 0x7f1054104b30 2024-04-24T23:59:59.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105c2bd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f105410d150 con 0x7f1054107460 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105c2bd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f105410d450 con 0x7f1054102200 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f1059858640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054104b30 0x7f1054197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f1059858640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054104b30 0x7f1054197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58110/0 (socket says 172.21.15.110:58110) 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105a85a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1054107460 0x7f105419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f1059858640 1 -- 172.21.15.110:0/4052408358 learned_addr learned my addr 172.21.15.110:0/4052408358 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.790+0000 7f105a059640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1054102200 0x7f1054197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f1059858640 1 -- 172.21.15.110:0/4052408358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1054102200 msgr2=0x7f1054197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f1059858640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1054102200 0x7f1054197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f1059858640 1 -- 172.21.15.110:0/4052408358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1054107460 msgr2=0x7f105419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f1059858640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1054107460 0x7f105419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f1059858640 1 -- 172.21.15.110:0/4052408358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105419f5a0 con 0x7f1054104b30 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f105a85a640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1054107460 0x7f105419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:59:59.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f1059858640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054104b30 0x7f1054197ec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f104400e9f0 tx=0x7f104400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:59.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f104b7fe640 1 -- 172.21.15.110:0/4052408358 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104400ccc0 con 0x7f1054104b30 2024-04-24T23:59:59.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105419f890 con 0x7f1054104b30 2024-04-24T23:59:59.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f104b7fe640 1 -- 172.21.15.110:0/4052408358 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10440054d0 con 0x7f1054104b30 2024-04-24T23:59:59.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f104b7fe640 1 -- 172.21.15.110:0/4052408358 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044010430 con 0x7f1054104b30 2024-04-24T23:59:59.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.791+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105419fde0 con 0x7f1054104b30 2024-04-24T23:59:59.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.792+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10540fd6e0 con 0x7f1054104b30 2024-04-24T23:59:59.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.794+0000 7f104b7fe640 1 -- 172.21.15.110:0/4052408358 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1044005000 con 0x7f1054104b30 2024-04-24T23:59:59.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.795+0000 7f104b7fe640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10400a2b90 0x7f10400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:59:59.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.795+0000 7f104b7fe640 1 -- 172.21.15.110:0/4052408358 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f10440cf170 con 0x7f1054104b30 2024-04-24T23:59:59.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.795+0000 7f105a059640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10400a2b90 0x7f10400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:59:59.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.796+0000 7f105a059640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10400a2b90 0x7f10400a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1050009bd0 tx=0x7f1050005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:59:59.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-24T23:59:59.801+0000 7f104b7fe640 1 -- 172.21.15.110:0/4052408358 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1044053740 con 0x7f1054104b30 2024-04-25T00:00:00.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.012+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f105402d370 con 0x7f1054104b30 2024-04-25T00:00:00.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.013+0000 7f104b7fe640 1 -- 172.21.15.110:0/4052408358 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18504 (secure 0 0 0) 0x7f10440052f0 con 0x7f1054104b30 2024-04-25T00:00:00.014 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:00.014 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":81,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-24T23:59:13.658286+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.015+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10400a2b90 msgr2=0x7f10400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.015+0000 7f105c2bd640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10400a2b90 0x7f10400a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1050009bd0 tx=0x7f1050005f90 comp rx=0 tx=0).stop 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.015+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054104b30 msgr2=0x7f1054197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.015+0000 7f105c2bd640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054104b30 0x7f1054197ec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f104400e9f0 tx=0x7f104400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.015+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 shutdown_connections 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.015+0000 7f105c2bd640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10400a2b90 0x7f10400a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.016+0000 7f105c2bd640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1054107460 0x7f105419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.016+0000 7f105c2bd640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1054104b30 0x7f1054197ec0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.016+0000 7f105c2bd640 1 --2- 172.21.15.110:0/4052408358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1054102200 0x7f1054197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.016+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 >> 172.21.15.110:0/4052408358 conn(0x7f10540fbbf0 msgr2=0x7f1054102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:00.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.016+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 shutdown_connections 2024-04-25T00:00:00.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.016+0000 7f105c2bd640 1 -- 172.21.15.110:0/4052408358 wait complete. 2024-04-25T00:00:00.027 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:00:00.027 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.133+0000 7f979dc7d640 1 -- 172.21.15.110:0/741675083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97981033d0 msgr2=0x7f9798109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.133+0000 7f979dc7d640 1 --2- 172.21.15.110:0/741675083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97981033d0 0x7f9798109c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9790009cd0 tx=0x7f979002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.133+0000 7f979dc7d640 1 -- 172.21.15.110:0/741675083 shutdown_connections 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.133+0000 7f979dc7d640 1 --2- 172.21.15.110:0/741675083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97981033d0 0x7f9798109c60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.133+0000 7f979dc7d640 1 --2- 172.21.15.110:0/741675083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9798102a10 0x7f9798102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.133+0000 7f979dc7d640 1 --2- 172.21.15.110:0/741675083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97980ffa40 0x7f97980ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.133+0000 7f979dc7d640 1 -- 172.21.15.110:0/741675083 >> 172.21.15.110:0/741675083 conn(0x7f97980fb370 msgr2=0x7f97980fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 -- 172.21.15.110:0/741675083 shutdown_connections 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 -- 172.21.15.110:0/741675083 wait complete. 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 Processor -- start 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 -- start start 2024-04-25T00:00:00.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97980ffa40 0x7f97981979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9798102a10 0x7f9798197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97981033d0 0x7f979819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f979810d2a0 con 0x7f97980ffa40 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f979810d120 con 0x7f97981033d0 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.134+0000 7f979dc7d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f979810d420 con 0x7f9798102a10 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97980ffa40 0x7f97981979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97980ffa40 0x7f97981979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58122/0 (socket says 172.21.15.110:58122) 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 -- 172.21.15.110:0/1021705085 learned_addr learned my addr 172.21.15.110:0/1021705085 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f9797fff640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97981033d0 0x7f979819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f9796ffd640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9798102a10 0x7f9798197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 -- 172.21.15.110:0/1021705085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9798102a10 msgr2=0x7f9798197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9798102a10 0x7f9798197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 -- 172.21.15.110:0/1021705085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97981033d0 msgr2=0x7f979819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97981033d0 0x7f979819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 -- 172.21.15.110:0/1021705085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979819f5e0 con 0x7f97980ffa40 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f9796ffd640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9798102a10 0x7f9798197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f9797fff640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97981033d0 0x7f979819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f97977fe640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97980ffa40 0x7f97981979c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f978800e9c0 tx=0x7f978800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f9794ff9640 1 -- 172.21.15.110:0/1021705085 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978800cd80 con 0x7f97980ffa40 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979819f8d0 con 0x7f97980ffa40 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f9794ff9640 1 -- 172.21.15.110:0/1021705085 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97880035f0 con 0x7f97980ffa40 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f9794ff9640 1 -- 172.21.15.110:0/1021705085 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788010690 con 0x7f97980ffa40 2024-04-25T00:00:00.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.135+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979819fe20 con 0x7f97980ffa40 2024-04-25T00:00:00.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.137+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9798108c10 con 0x7f97980ffa40 2024-04-25T00:00:00.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.137+0000 7f9794ff9640 1 -- 172.21.15.110:0/1021705085 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f97880037a0 con 0x7f97980ffa40 2024-04-25T00:00:00.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.138+0000 7f9794ff9640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97800a2b90 0x7f97800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.138+0000 7f9796ffd640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97800a2b90 0x7f97800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.139+0000 7f9794ff9640 1 -- 172.21.15.110:0/1021705085 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f97880cefe0 con 0x7f97980ffa40 2024-04-25T00:00:00.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.139+0000 7f9796ffd640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97800a2b90 0x7f97800a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9798198ed0 tx=0x7f978c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:00.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.143+0000 7f9794ff9640 1 -- 172.21.15.110:0/1021705085 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f978809c340 con 0x7f97980ffa40 2024-04-25T00:00:00.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.336+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f979802d170 con 0x7f97980ffa40 2024-04-25T00:00:00.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.337+0000 7f9794ff9640 1 -- 172.21.15.110:0/1021705085 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f979802d170 con 0x7f97980ffa40 2024-04-25T00:00:00.337 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T00:00:00.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97800a2b90 msgr2=0x7f97800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97800a2b90 0x7f97800a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9798198ed0 tx=0x7f978c005f90 comp rx=0 tx=0).stop 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97980ffa40 msgr2=0x7f97981979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97980ffa40 0x7f97981979c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f978800e9c0 tx=0x7f978800ee80 comp rx=0 tx=0).stop 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 shutdown_connections 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97800a2b90 0x7f97800a5050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97981033d0 0x7f979819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9798102a10 0x7f9798197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 --2- 172.21.15.110:0/1021705085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97980ffa40 0x7f97981979c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 >> 172.21.15.110:0/1021705085 conn(0x7f97980fb370 msgr2=0x7f9798105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 shutdown_connections 2024-04-25T00:00:00.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.339+0000 7f979dc7d640 1 -- 172.21.15.110:0/1021705085 wait complete. 2024-04-25T00:00:00.351 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:00:00.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 -- 172.21.15.110:0/3541910093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b54102240 msgr2=0x7f5b54108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 --2- 172.21.15.110:0/3541910093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b54102240 0x7f5b54108ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009ca0 tx=0x7f5b4c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 -- 172.21.15.110:0/3541910093 shutdown_connections 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 --2- 172.21.15.110:0/3541910093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b54102240 0x7f5b54108ad0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 --2- 172.21.15.110:0/3541910093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54101880 0x7f5b54101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 --2- 172.21.15.110:0/3541910093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b54100690 0x7f5b54100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 -- 172.21.15.110:0/3541910093 >> 172.21.15.110:0/3541910093 conn(0x7f5b540fbc20 msgr2=0x7f5b540fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 -- 172.21.15.110:0/3541910093 shutdown_connections 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 -- 172.21.15.110:0/3541910093 wait complete. 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 Processor -- start 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.452+0000 7f5b59509640 1 -- start start 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b59509640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b54100690 0x7f5b5406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b59509640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b54101880 0x7f5b5419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b59509640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54102240 0x7f5b5419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b59509640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410c1a0 con 0x7f5b54102240 2024-04-25T00:00:00.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b59509640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410c020 con 0x7f5b54100690 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b59509640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410c320 con 0x7f5b54101880 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54102240 0x7f5b5419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54102240 0x7f5b5419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58138/0 (socket says 172.21.15.110:58138) 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 -- 172.21.15.110:0/4262076041 learned_addr learned my addr 172.21.15.110:0/4262076041 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b52ffd640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b54100690 0x7f5b5406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b527fc640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b54101880 0x7f5b5419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 -- 172.21.15.110:0/4262076041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b54101880 msgr2=0x7f5b5419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b54101880 0x7f5b5419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 -- 172.21.15.110:0/4262076041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b54100690 msgr2=0x7f5b5406a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b54100690 0x7f5b5406a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 -- 172.21.15.110:0/4262076041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5419f5c0 con 0x7f5b54102240 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b52ffd640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b54100690 0x7f5b5406a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b537fe640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54102240 0x7f5b5419ecb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c031970 tx=0x7f5b4c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b3bfff640 1 -- 172.21.15.110:0/4262076041 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c003d90 con 0x7f5b54102240 2024-04-25T00:00:00.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5419f7f0 con 0x7f5b54102240 2024-04-25T00:00:00.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b3bfff640 1 -- 172.21.15.110:0/4262076041 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b4c0044f0 con 0x7f5b54102240 2024-04-25T00:00:00.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b3bfff640 1 -- 172.21.15.110:0/4262076041 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c030bd0 con 0x7f5b54102240 2024-04-25T00:00:00.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.453+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5419fd90 con 0x7f5b54102240 2024-04-25T00:00:00.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.454+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b541079b0 con 0x7f5b54102240 2024-04-25T00:00:00.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.455+0000 7f5b3bfff640 1 -- 172.21.15.110:0/4262076041 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5b4c030d70 con 0x7f5b54102240 2024-04-25T00:00:00.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.456+0000 7f5b3bfff640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b3c0a2b90 0x7f5b3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.456+0000 7f5b52ffd640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b3c0a2b90 0x7f5b3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.456+0000 7f5b3bfff640 1 -- 172.21.15.110:0/4262076041 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5b4c0f0110 con 0x7f5b54102240 2024-04-25T00:00:00.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.457+0000 7f5b52ffd640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b3c0a2b90 0x7f5b3c0a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b541016e0 tx=0x7f5b44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:00.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.460+0000 7f5b3bfff640 1 -- 172.21.15.110:0/4262076041 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5b4c0bd510 con 0x7f5b54102240 2024-04-25T00:00:00.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.670+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5b5402d2f0 con 0x7f5b54102240 2024-04-25T00:00:00.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.671+0000 7f5b3bfff640 1 -- 172.21.15.110:0/4262076041 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f5b4c0bce10 con 0x7f5b54102240 2024-04-25T00:00:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:00.671 INFO:teuthology.orchestra.run.smithi110.stdout:[ 2024-04-25T00:00:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:00:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "poolnum": 1, 2024-04-25T00:00:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "poolname": ".mgr" 2024-04-25T00:00:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:00:00.672 INFO:teuthology.orchestra.run.smithi110.stdout:] 2024-04-25T00:00:00.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b3c0a2b90 msgr2=0x7f5b3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b3c0a2b90 0x7f5b3c0a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b541016e0 tx=0x7f5b44005f90 comp rx=0 tx=0).stop 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54102240 msgr2=0x7f5b5419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54102240 0x7f5b5419ecb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c031970 tx=0x7f5b4c003ba0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 shutdown_connections 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b3c0a2b90 0x7f5b3c0a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54102240 0x7f5b5419ecb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b54101880 0x7f5b5419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 --2- 172.21.15.110:0/4262076041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b54100690 0x7f5b5406a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 >> 172.21.15.110:0/4262076041 conn(0x7f5b540fbc20 msgr2=0x7f5b54104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 shutdown_connections 2024-04-25T00:00:00.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.673+0000 7f5b59509640 1 -- 172.21.15.110:0/4262076041 wait complete. 2024-04-25T00:00:00.684 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.684 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4073342749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: overall HEALTH_OK 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4052408358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.685 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:00 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1021705085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:00:00.686 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:00:00.687 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:00:00.756 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.756 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4073342749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.756 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.756 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: overall HEALTH_OK 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4052408358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.757 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:00 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1021705085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:00:00.762 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.762 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4073342749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.762 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.762 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.762 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.762 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: overall HEALTH_OK 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4052408358' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:00:00.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:00 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1021705085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:00:00.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 -- 172.21.15.110:0/2445562555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff50102a20 msgr2=0x7eff50102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 --2- 172.21.15.110:0/2445562555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff50102a20 0x7eff50102ea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009930 tx=0x7eff3c02a420 comp rx=0 tx=0).stop 2024-04-25T00:00:00.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 -- 172.21.15.110:0/2445562555 shutdown_connections 2024-04-25T00:00:00.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 --2- 172.21.15.110:0/2445562555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff501033e0 0x7eff50109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 --2- 172.21.15.110:0/2445562555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff50102a20 0x7eff50102ea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 --2- 172.21.15.110:0/2445562555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500ffa50 0x7eff500ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 -- 172.21.15.110:0/2445562555 >> 172.21.15.110:0/2445562555 conn(0x7eff500fb380 msgr2=0x7eff500fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 -- 172.21.15.110:0/2445562555 shutdown_connections 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.792+0000 7eff56d16640 1 -- 172.21.15.110:0/2445562555 wait complete. 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff56d16640 1 Processor -- start 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff56d16640 1 -- start start 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff56d16640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500ffa50 0x7eff50102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff56d16640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff50102a20 0x7eff501009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff56d16640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff501033e0 0x7eff50100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff56d16640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7eff5010d330 con 0x7eff501033e0 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff56d16640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff5010d1b0 con 0x7eff500ffa50 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff56d16640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7eff5010d4b0 con 0x7eff50102a20 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff501033e0 0x7eff50100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff54ab2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500ffa50 0x7eff50102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff501033e0 0x7eff50100f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58164/0 (socket says 172.21.15.110:58164) 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 -- 172.21.15.110:0/2114850137 learned_addr learned my addr 172.21.15.110:0/2114850137 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff4ffff640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff50102a20 0x7eff501009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 -- 172.21.15.110:0/2114850137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff50102a20 msgr2=0x7eff501009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff50102a20 0x7eff501009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 -- 172.21.15.110:0/2114850137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500ffa50 msgr2=0x7eff50102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500ffa50 0x7eff50102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 -- 172.21.15.110:0/2114850137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff501017a0 con 0x7eff501033e0 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.793+0000 7eff552b3640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff501033e0 0x7eff50100f10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff4800eb10 tx=0x7eff4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.794+0000 7eff4dffb640 1 -- 172.21.15.110:0/2114850137 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48004b20 con 0x7eff501033e0 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.794+0000 7eff4dffb640 1 -- 172.21.15.110:0/2114850137 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff48004cc0 con 0x7eff501033e0 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.794+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff501a1b20 con 0x7eff501033e0 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.794+0000 7eff4dffb640 1 -- 172.21.15.110:0/2114850137 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48010690 con 0x7eff501033e0 2024-04-25T00:00:00.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.794+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501a2040 con 0x7eff501033e0 2024-04-25T00:00:00.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.795+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff500689b0 con 0x7eff501033e0 2024-04-25T00:00:00.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.796+0000 7eff4dffb640 1 -- 172.21.15.110:0/2114850137 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7eff48005020 con 0x7eff501033e0 2024-04-25T00:00:00.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.796+0000 7eff4dffb640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7eff400a2b90 0x7eff400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:00.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.797+0000 7eff54ab2640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7eff400a2b90 0x7eff400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:00.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.797+0000 7eff4dffb640 1 -- 172.21.15.110:0/2114850137 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7eff480cf2d0 con 0x7eff501033e0 2024-04-25T00:00:00.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.797+0000 7eff54ab2640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7eff400a2b90 0x7eff400a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff50069bc0 tx=0x7eff44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:00.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.801+0000 7eff4dffb640 1 -- 172.21.15.110:0/2114850137 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff4809c6d0 con 0x7eff501033e0 2024-04-25T00:00:00.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:00.982+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7eff50062340 con 0x7eff501033e0 2024-04-25T00:00:01.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.505+0000 7eff4dffb640 1 -- 172.21.15.110:0/2114850137 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7eff4809bfd0 con 0x7eff501033e0 2024-04-25T00:00:01.506 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' created 2024-04-25T00:00:01.507 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:01 smithi173 ceph-mon[32402]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:00:01.507 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:01 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4262076041' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:00:01.507 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:01 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2114850137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:00:01.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7eff400a2b90 msgr2=0x7eff400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7eff400a2b90 0x7eff400a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff50069bc0 tx=0x7eff44005f90 comp rx=0 tx=0).stop 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff501033e0 msgr2=0x7eff50100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff501033e0 0x7eff50100f10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff4800eb10 tx=0x7eff4800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 shutdown_connections 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7eff400a2b90 0x7eff400a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff501033e0 0x7eff50100f10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff50102a20 0x7eff501009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 --2- 172.21.15.110:0/2114850137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500ffa50 0x7eff50102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 >> 172.21.15.110:0/2114850137 conn(0x7eff500fb380 msgr2=0x7eff50105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:01.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 shutdown_connections 2024-04-25T00:00:01.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.508+0000 7eff56d16640 1 -- 172.21.15.110:0/2114850137 wait complete. 2024-04-25T00:00:01.524 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:00:01.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 -- 172.21.15.110:0/4156079520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4104b30 msgr2=0x7f8da4106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:01.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 --2- 172.21.15.110:0/4156079520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4104b30 0x7f8da4106f20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009900 tx=0x7f8d8c02a410 comp rx=0 tx=0).stop 2024-04-25T00:00:01.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 -- 172.21.15.110:0/4156079520 shutdown_connections 2024-04-25T00:00:01.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 --2- 172.21.15.110:0/4156079520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8da4107460 0x7f8da41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 --2- 172.21.15.110:0/4156079520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4104b30 0x7f8da4106f20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 --2- 172.21.15.110:0/4156079520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8da4102200 0x7f8da41045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 -- 172.21.15.110:0/4156079520 >> 172.21.15.110:0/4156079520 conn(0x7f8da40fbbf0 msgr2=0x7f8da40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:01.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 -- 172.21.15.110:0/4156079520 shutdown_connections 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 -- 172.21.15.110:0/4156079520 wait complete. 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 Processor -- start 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.637+0000 7f8da8a12640 1 -- start start 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da8a12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8da4102200 0x7f8da4199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da8a12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8da4104b30 0x7f8da419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da8a12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4107460 0x7f8da41a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da8a12640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8da410d2c0 con 0x7f8da4107460 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da8a12640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8da410d140 con 0x7f8da4104b30 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da8a12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8da410d440 con 0x7f8da4102200 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4107460 0x7f8da41a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4107460 0x7f8da41a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58180/0 (socket says 172.21.15.110:58180) 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8da4102200 0x7f8da4199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:01.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 -- 172.21.15.110:0/1622118432 learned_addr learned my addr 172.21.15.110:0/1622118432 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:01.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8d9bfff640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8da4104b30 0x7f8da419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:01.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 -- 172.21.15.110:0/1622118432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8da4102200 msgr2=0x7f8da4199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:01.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8da4102200 0x7f8da4199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 -- 172.21.15.110:0/1622118432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8da4104b30 msgr2=0x7f8da419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:01.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8da4104b30 0x7f8da419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:01.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 -- 172.21.15.110:0/1622118432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da41a1840 con 0x7f8da4107460 2024-04-25T00:00:01.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da2d9d640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4107460 0x7f8da41a11a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00eaa0 tx=0x7f8d9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:01.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8d9b7fe640 1 -- 172.21.15.110:0/1622118432 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c00cdb0 con 0x7f8da4107460 2024-04-25T00:00:01.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.638+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da41a1b30 con 0x7f8da4107460 2024-04-25T00:00:01.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.639+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da41a2080 con 0x7f8da4107460 2024-04-25T00:00:01.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.639+0000 7f8d9b7fe640 1 -- 172.21.15.110:0/1622118432 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d9c005020 con 0x7f8da4107460 2024-04-25T00:00:01.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.639+0000 7f8d9b7fe640 1 -- 172.21.15.110:0/1622118432 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c01f060 con 0x7f8da4107460 2024-04-25T00:00:01.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.639+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da4108fb0 con 0x7f8da4107460 2024-04-25T00:00:01.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.641+0000 7f8d9b7fe640 1 -- 172.21.15.110:0/1622118432 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8d9c0051d0 con 0x7f8da4107460 2024-04-25T00:00:01.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.641+0000 7f8d9b7fe640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8d900a2b90 0x7f8d900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:01.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.642+0000 7f8d9b7fe640 1 -- 172.21.15.110:0/1622118432 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f8d9c0cf260 con 0x7f8da4107460 2024-04-25T00:00:01.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.641+0000 7f8da259c640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8d900a2b90 0x7f8d900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:01.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.642+0000 7f8da259c640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8d900a2b90 0x7f8d900a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8da419b100 tx=0x7f8d94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:01.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.649+0000 7f8d9b7fe640 1 -- 172.21.15.110:0/1622118432 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8d9c016030 con 0x7f8da4107460 2024-04-25T00:00:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:01 smithi188 ceph-mon[33285]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:00:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:01 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4262076041' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:00:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:01 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2114850137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:00:01.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:01 smithi110 ceph-mon[29187]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:00:01.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:01 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4262076041' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:00:01.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:01 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2114850137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:00:01.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:01.858+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8da4062340 con 0x7f8da4107460 2024-04-25T00:00:02.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.507+0000 7f8d9b7fe640 1 -- 172.21.15.110:0/1622118432 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f8d9c09be80 con 0x7f8da4107460 2024-04-25T00:00:02.508 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' created 2024-04-25T00:00:02.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8d900a2b90 msgr2=0x7f8d900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8d900a2b90 0x7f8d900a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8da419b100 tx=0x7f8d94005f90 comp rx=0 tx=0).stop 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4107460 msgr2=0x7f8da41a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4107460 0x7f8da41a11a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00eaa0 tx=0x7f8d9c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 shutdown_connections 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8d900a2b90 0x7f8d900a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8da4107460 0x7f8da41a11a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8da4104b30 0x7f8da419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 --2- 172.21.15.110:0/1622118432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8da4102200 0x7f8da4199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 >> 172.21.15.110:0/1622118432 conn(0x7f8da40fbbf0 msgr2=0x7f8da4101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 shutdown_connections 2024-04-25T00:00:02.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.510+0000 7f8da8a12640 1 -- 172.21.15.110:0/1622118432 wait complete. 2024-04-25T00:00:02.530 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 -- 172.21.15.110:0/3798061445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b501012b0 msgr2=0x7f9b50108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3798061445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b501012b0 0x7f9b50108b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009ca0 tx=0x7f9b4802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 -- 172.21.15.110:0/3798061445 shutdown_connections 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3798061445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b501012b0 0x7f9b50108b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3798061445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b501008f0 0x7f9b50100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3798061445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b50101ca0 0x7f9b501020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 -- 172.21.15.110:0/3798061445 >> 172.21.15.110:0/3798061445 conn(0x7f9b500fb380 msgr2=0x7f9b500fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 -- 172.21.15.110:0/3798061445 shutdown_connections 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.632+0000 7f9b5691d640 1 -- 172.21.15.110:0/3798061445 wait complete. 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b5691d640 1 Processor -- start 2024-04-25T00:00:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b5691d640 1 -- start start 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b5691d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b501008f0 0x7f9b50100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b5691d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b501012b0 0x7f9b5019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b5691d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b50101ca0 0x7f9b5019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b5691d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5010c1e0 con 0x7f9b50101ca0 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b5691d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5010c060 con 0x7f9b501008f0 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b5691d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5010c360 con 0x7f9b501012b0 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b54eba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b50101ca0 0x7f9b5019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b54eba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b50101ca0 0x7f9b5019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58194/0 (socket says 172.21.15.110:58194) 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b54eba640 1 -- 172.21.15.110:0/3814088408 learned_addr learned my addr 172.21.15.110:0/3814088408 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b4f7fe640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b501012b0 0x7f9b5019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.633+0000 7f9b4ffff640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b501008f0 0x7f9b50100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b54eba640 1 -- 172.21.15.110:0/3814088408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b501012b0 msgr2=0x7f9b5019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b54eba640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b501012b0 0x7f9b5019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b54eba640 1 -- 172.21.15.110:0/3814088408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b501008f0 msgr2=0x7f9b50100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b54eba640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b501008f0 0x7f9b50100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b54eba640 1 -- 172.21.15.110:0/3814088408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5019f600 con 0x7f9b50101ca0 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b54eba640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b50101ca0 0x7f9b5019ecf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b48031970 tx=0x7f9b48003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b4d7fa640 1 -- 172.21.15.110:0/3814088408 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48004040 con 0x7f9b50101ca0 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b4d7fa640 1 -- 172.21.15.110:0/3814088408 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b48004690 con 0x7f9b50101ca0 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5019f830 con 0x7f9b50101ca0 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b4d7fa640 1 -- 172.21.15.110:0/3814088408 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4803c410 con 0x7f9b50101ca0 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.634+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5019fd80 con 0x7f9b50101ca0 2024-04-25T00:00:02.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.635+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b50107ad0 con 0x7f9b50101ca0 2024-04-25T00:00:02.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.636+0000 7f9b4d7fa640 1 -- 172.21.15.110:0/3814088408 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9b480305c0 con 0x7f9b50101ca0 2024-04-25T00:00:02.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.636+0000 7f9b4d7fa640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9b380a2b90 0x7f9b380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:02.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.637+0000 7f9b4ffff640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9b380a2b90 0x7f9b380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:02.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.637+0000 7f9b4d7fa640 1 -- 172.21.15.110:0/3814088408 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f9b480f0720 con 0x7f9b50101ca0 2024-04-25T00:00:02.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.637+0000 7f9b4ffff640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9b380a2b90 0x7f9b380a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9b50069ba0 tx=0x7f9b40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.640+0000 7f9b4d7fa640 1 -- 172.21.15.110:0/3814088408 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9b480bd830 con 0x7f9b50101ca0 2024-04-25T00:00:02.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2114850137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:00:02.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:00:02.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1622118432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:00:02.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:00:02.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:00:02.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2114850137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:00:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:00:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1622118432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:00:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:00:02.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:00:02.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:02.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:02.854+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9b500623b0 con 0x7f9b50101ca0 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2114850137' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1622118432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:03.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:00:03.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:00:03.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:00:03.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.518+0000 7f9b4d7fa640 1 -- 172.21.15.110:0/3814088408 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f9b480bd130 con 0x7f9b50101ca0 2024-04-25T00:00:03.518 INFO:teuthology.orchestra.run.smithi110.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:00:03.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9b380a2b90 msgr2=0x7f9b380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9b380a2b90 0x7f9b380a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9b50069ba0 tx=0x7f9b40005f90 comp rx=0 tx=0).stop 2024-04-25T00:00:03.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b50101ca0 msgr2=0x7f9b5019ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b50101ca0 0x7f9b5019ecf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b48031970 tx=0x7f9b48003ba0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 shutdown_connections 2024-04-25T00:00:03.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9b380a2b90 0x7f9b380a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b50101ca0 0x7f9b5019ecf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b501012b0 0x7f9b5019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 --2- 172.21.15.110:0/3814088408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b501008f0 0x7f9b50100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 >> 172.21.15.110:0/3814088408 conn(0x7f9b500fb380 msgr2=0x7f9b50104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 shutdown_connections 2024-04-25T00:00:03.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.520+0000 7f9b5691d640 1 -- 172.21.15.110:0/3814088408 wait complete. 2024-04-25T00:00:03.535 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:03.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.631+0000 7f37b384c640 1 -- 172.21.15.110:0/2861372988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac103340 msgr2=0x7f37ac109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.631+0000 7f37b384c640 1 --2- 172.21.15.110:0/2861372988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac103340 0x7f37ac109bd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009900 tx=0x7f37a402a410 comp rx=0 tx=0).stop 2024-04-25T00:00:03.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 -- 172.21.15.110:0/2861372988 shutdown_connections 2024-04-25T00:00:03.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 --2- 172.21.15.110:0/2861372988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac103340 0x7f37ac109bd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 --2- 172.21.15.110:0/2861372988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ac102980 0x7f37ac102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 --2- 172.21.15.110:0/2861372988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37ac101790 0x7f37ac101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 -- 172.21.15.110:0/2861372988 >> 172.21.15.110:0/2861372988 conn(0x7f37ac0fcd20 msgr2=0x7f37ac0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 -- 172.21.15.110:0/2861372988 shutdown_connections 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 -- 172.21.15.110:0/2861372988 wait complete. 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 Processor -- start 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 -- start start 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac101790 0x7f37ac195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37ac102980 0x7f37ac195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ac103340 0x7f37ac19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10b0e0 con 0x7f37ac101790 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10af60 con 0x7f37ac103340 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b384c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10b260 con 0x7f37ac102980 2024-04-25T00:00:03.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b15e8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac101790 0x7f37ac195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b15e8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac101790 0x7f37ac195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58208/0 (socket says 172.21.15.110:58208) 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.632+0000 7f37b15e8640 1 -- 172.21.15.110:0/3058580047 learned_addr learned my addr 172.21.15.110:0/3058580047 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b0de7640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37ac102980 0x7f37ac195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b1de9640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ac103340 0x7f37ac19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b15e8640 1 -- 172.21.15.110:0/3058580047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37ac102980 msgr2=0x7f37ac195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b15e8640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37ac102980 0x7f37ac195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b15e8640 1 -- 172.21.15.110:0/3058580047 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ac103340 msgr2=0x7f37ac19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b15e8640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ac103340 0x7f37ac19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b15e8640 1 -- 172.21.15.110:0/3058580047 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac19d360 con 0x7f37ac101790 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b15e8640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac101790 0x7f37ac195740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37a800eaa0 tx=0x7f37a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37a27fc640 1 -- 172.21.15.110:0/3058580047 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8016070 con 0x7f37ac101790 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37a27fc640 1 -- 172.21.15.110:0/3058580047 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37a8003e20 con 0x7f37ac101790 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac19d650 con 0x7f37ac101790 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37a27fc640 1 -- 172.21.15.110:0/3058580047 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80116d0 con 0x7f37ac101790 2024-04-25T00:00:03.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.633+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac19dba0 con 0x7f37ac101790 2024-04-25T00:00:03.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.634+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac108b30 con 0x7f37ac101790 2024-04-25T00:00:03.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.635+0000 7f37a27fc640 1 -- 172.21.15.110:0/3058580047 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f37a8019070 con 0x7f37ac101790 2024-04-25T00:00:03.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.636+0000 7f37a27fc640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37980a2c10 0x7f37980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.636+0000 7f37b0de7640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37980a2c10 0x7f37980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.636+0000 7f37a27fc640 1 -- 172.21.15.110:0/3058580047 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f37a80cf3f0 con 0x7f37ac101790 2024-04-25T00:00:03.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.636+0000 7f37b0de7640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37980a2c10 0x7f37980a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37ac196c50 tx=0x7f379c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.639+0000 7f37a27fc640 1 -- 172.21.15.110:0/3058580047 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f37a809c520 con 0x7f37ac101790 2024-04-25T00:00:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:03 smithi110 ceph-mon[29187]: pgmap v286: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:00:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:03 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1622118432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:00:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:03 smithi110 ceph-mon[29187]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:00:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:03 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3814088408' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:00:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:03 smithi110 ceph-mon[29187]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:00:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:03 smithi110 ceph-mon[29187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:00:03.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:03 smithi110 ceph-mon[29187]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:00:03.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.819+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37ac062370 con 0x7f37ac101790 2024-04-25T00:00:03.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.821+0000 7f37a27fc640 1 -- 172.21.15.110:0/3058580047 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22280 (secure 0 0 0) 0x7f37a809be20 con 0x7f37ac101790 2024-04-25T00:00:03.821 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:03.821 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":84,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:02.856193+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:03.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37980a2c10 msgr2=0x7f37980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37980a2c10 0x7f37980a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37ac196c50 tx=0x7f379c005f90 comp rx=0 tx=0).stop 2024-04-25T00:00:03.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac101790 msgr2=0x7f37ac195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac101790 0x7f37ac195740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37a800eaa0 tx=0x7f37a800ef60 comp rx=0 tx=0).stop 2024-04-25T00:00:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 shutdown_connections 2024-04-25T00:00:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37980a2c10 0x7f37980a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37ac103340 0x7f37ac19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37ac102980 0x7f37ac195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 --2- 172.21.15.110:0/3058580047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37ac101790 0x7f37ac195740 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 >> 172.21.15.110:0/3058580047 conn(0x7f37ac0fcd20 msgr2=0x7f37ac105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 shutdown_connections 2024-04-25T00:00:03.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.823+0000 7f37b384c640 1 -- 172.21.15.110:0/3058580047 wait complete. 2024-04-25T00:00:03.834 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:03.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:03 smithi173 ceph-mon[32402]: pgmap v286: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:00:03.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:03 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1622118432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:00:03.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:03 smithi173 ceph-mon[32402]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:00:03.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:03 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3814088408' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:00:03.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:03 smithi173 ceph-mon[32402]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:00:03.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:03 smithi173 ceph-mon[32402]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:00:03.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:03 smithi173 ceph-mon[32402]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:00:03.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.932+0000 7f788062a640 1 -- 172.21.15.110:0/2599379373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 msgr2=0x7f7878100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.932+0000 7f788062a640 1 --2- 172.21.15.110:0/2599379373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 0x7f7878100ac0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7868009900 tx=0x7f786802a410 comp rx=0 tx=0).stop 2024-04-25T00:00:03.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.932+0000 7f788062a640 1 -- 172.21.15.110:0/2599379373 shutdown_connections 2024-04-25T00:00:03.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.932+0000 7f788062a640 1 --2- 172.21.15.110:0/2599379373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7878102250 0x7f7878108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.932+0000 7f788062a640 1 --2- 172.21.15.110:0/2599379373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7878101890 0x7f7878101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.932+0000 7f788062a640 1 --2- 172.21.15.110:0/2599379373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 0x7f7878100ac0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.932+0000 7f788062a640 1 -- 172.21.15.110:0/2599379373 >> 172.21.15.110:0/2599379373 conn(0x7f78780fbc30 msgr2=0x7f78780fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:03.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 -- 172.21.15.110:0/2599379373 shutdown_connections 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 -- 172.21.15.110:0/2599379373 wait complete. 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 Processor -- start 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 -- start start 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 0x7f787806a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7878101890 0x7f787819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7878102250 0x7f787819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f787810c1d0 con 0x7f78781006a0 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f787810c050 con 0x7f7878102250 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f788062a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f787810c350 con 0x7f7878101890 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f787e3c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 0x7f787806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f787e3c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 0x7f787806a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58214/0 (socket says 172.21.15.110:58214) 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f787e3c6640 1 -- 172.21.15.110:0/234411857 learned_addr learned my addr 172.21.15.110:0/234411857 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.933+0000 7f787dbc5640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7878101890 0x7f787819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f787ebc7640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7878102250 0x7f787819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f787e3c6640 1 -- 172.21.15.110:0/234411857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7878101890 msgr2=0x7f787819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f787e3c6640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7878101890 0x7f787819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f787e3c6640 1 -- 172.21.15.110:0/234411857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7878102250 msgr2=0x7f787819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f787e3c6640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7878102250 0x7f787819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f787e3c6640 1 -- 172.21.15.110:0/234411857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787819f640 con 0x7f78781006a0 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f787ebc7640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7878102250 0x7f787819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f787e3c6640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 0x7f787806a950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f78680314e0 tx=0x7f78680042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f786f7fe640 1 -- 172.21.15.110:0/234411857 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786803f030 con 0x7f78781006a0 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f786f7fe640 1 -- 172.21.15.110:0/234411857 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7868003cb0 con 0x7f78781006a0 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787819f8d0 con 0x7f78781006a0 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f786f7fe640 1 -- 172.21.15.110:0/234411857 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868035070 con 0x7f78781006a0 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.934+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787819fdc0 con 0x7f78781006a0 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.935+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7878107ae0 con 0x7f78781006a0 2024-04-25T00:00:03.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.936+0000 7f786f7fe640 1 -- 172.21.15.110:0/234411857 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7868042030 con 0x7f78781006a0 2024-04-25T00:00:03.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.937+0000 7f786f7fe640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f78640a2b90 0x7f78640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:03.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.937+0000 7f787dbc5640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f78640a2b90 0x7f78640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:03.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.937+0000 7f786f7fe640 1 -- 172.21.15.110:0/234411857 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f78680ef510 con 0x7f78781006a0 2024-04-25T00:00:03.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.937+0000 7f787dbc5640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f78640a2b90 0x7f78640a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f787806a6c0 tx=0x7f787400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:03.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:03.941+0000 7f786f7fe640 1 -- 172.21.15.110:0/234411857 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f78680bc5c0 con 0x7f78781006a0 2024-04-25T00:00:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:03 smithi188 ceph-mon[33285]: pgmap v286: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:00:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:03 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1622118432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:00:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:03 smithi188 ceph-mon[33285]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:00:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:03 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3814088408' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:00:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:03 smithi188 ceph-mon[33285]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:00:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:03 smithi188 ceph-mon[33285]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:00:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:03 smithi188 ceph-mon[33285]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:00:04.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.122+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f787802d3c0 con 0x7f78781006a0 2024-04-25T00:00:04.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.123+0000 7f786f7fe640 1 -- 172.21.15.110:0/234411857 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22280 (secure 0 0 0) 0x7f78680bbec0 con 0x7f78781006a0 2024-04-25T00:00:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:04.124 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":84,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:02.856193+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f78640a2b90 msgr2=0x7f78640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f78640a2b90 0x7f78640a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f787806a6c0 tx=0x7f787400b410 comp rx=0 tx=0).stop 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 msgr2=0x7f787806a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 0x7f787806a950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f78680314e0 tx=0x7f78680042e0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 shutdown_connections 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f78640a2b90 0x7f78640a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7878102250 0x7f787819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7878101890 0x7f787819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 --2- 172.21.15.110:0/234411857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78781006a0 0x7f787806a950 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 >> 172.21.15.110:0/234411857 conn(0x7f78780fbc30 msgr2=0x7f7878104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 shutdown_connections 2024-04-25T00:00:04.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.126+0000 7f788062a640 1 -- 172.21.15.110:0/234411857 wait complete. 2024-04-25T00:00:04.138 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 -- 172.21.15.110:0/326998664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c100680 msgr2=0x7fcc9c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/326998664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c100680 0x7fcc9c100aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009900 tx=0x7fcc8c02a410 comp rx=0 tx=0).stop 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 -- 172.21.15.110:0/326998664 shutdown_connections 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/326998664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc9c102230 0x7fcc9c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/326998664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9c101870 0x7fcc9c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/326998664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c100680 0x7fcc9c100aa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 -- 172.21.15.110:0/326998664 >> 172.21.15.110:0/326998664 conn(0x7fcc9c0fbbf0 msgr2=0x7fcc9c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 -- 172.21.15.110:0/326998664 shutdown_connections 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.240+0000 7fcca3bf9640 1 -- 172.21.15.110:0/326998664 wait complete. 2024-04-25T00:00:04.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca3bf9640 1 Processor -- start 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca3bf9640 1 -- start start 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca3bf9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc9c100680 0x7fcc9c199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca3bf9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9c101870 0x7fcc9c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca3bf9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c102230 0x7fcc9c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca3bf9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c10c1b0 con 0x7fcc9c102230 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca3bf9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c10c030 con 0x7fcc9c101870 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca3bf9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c10c330 con 0x7fcc9c100680 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca2196640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c102230 0x7fcc9c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca2196640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c102230 0x7fcc9c1a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58228/0 (socket says 172.21.15.110:58228) 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca2196640 1 -- 172.21.15.110:0/2867821574 learned_addr learned my addr 172.21.15.110:0/2867821574 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca1995640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc9c100680 0x7fcc9c199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca1194640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9c101870 0x7fcc9c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca2196640 1 -- 172.21.15.110:0/2867821574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc9c100680 msgr2=0x7fcc9c199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca2196640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc9c100680 0x7fcc9c199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca2196640 1 -- 172.21.15.110:0/2867821574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9c101870 msgr2=0x7fcc9c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca2196640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9c101870 0x7fcc9c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca2196640 1 -- 172.21.15.110:0/2867821574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c1a1830 con 0x7fcc9c102230 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.241+0000 7fcca1194640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9c101870 0x7fcc9c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.242+0000 7fcca2196640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c102230 0x7fcc9c1a1190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400eaa0 tx=0x7fcc9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.242+0000 7fcc92ffd640 1 -- 172.21.15.110:0/2867821574 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400cdb0 con 0x7fcc9c102230 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.242+0000 7fcc92ffd640 1 -- 172.21.15.110:0/2867821574 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc94005020 con 0x7fcc9c102230 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.242+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9c1a1b20 con 0x7fcc9c102230 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.242+0000 7fcc92ffd640 1 -- 172.21.15.110:0/2867821574 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940116d0 con 0x7fcc9c102230 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.242+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9c1a2040 con 0x7fcc9c102230 2024-04-25T00:00:04.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.243+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc9c1a2270 con 0x7fcc9c102230 2024-04-25T00:00:04.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.244+0000 7fcc92ffd640 1 -- 172.21.15.110:0/2867821574 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcc940051d0 con 0x7fcc9c102230 2024-04-25T00:00:04.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.244+0000 7fcc92ffd640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcc880a2b90 0x7fcc880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:04.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.245+0000 7fcca1995640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcc880a2b90 0x7fcc880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:04.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.245+0000 7fcc92ffd640 1 -- 172.21.15.110:0/2867821574 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fcc940cf500 con 0x7fcc9c102230 2024-04-25T00:00:04.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.245+0000 7fcca1995640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcc880a2b90 0x7fcc880a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c004770 tx=0x7fcc8c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:04.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.248+0000 7fcc92ffd640 1 -- 172.21.15.110:0/2867821574 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc94016030 con 0x7fcc9c102230 2024-04-25T00:00:04.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.461+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcc9c1a2660 con 0x7fcc9c102230 2024-04-25T00:00:04.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.527+0000 7fcc92ffd640 1 -- 172.21.15.110:0/2867821574 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fcc9409c5b0 con 0x7fcc9c102230 2024-04-25T00:00:04.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcc880a2b90 msgr2=0x7fcc880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcc880a2b90 0x7fcc880a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c004770 tx=0x7fcc8c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c102230 msgr2=0x7fcc9c1a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c102230 0x7fcc9c1a1190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400eaa0 tx=0x7fcc9400ef60 comp rx=0 tx=0).stop 2024-04-25T00:00:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 shutdown_connections 2024-04-25T00:00:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcc880a2b90 0x7fcc880a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc9c102230 0x7fcc9c1a1190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9c101870 0x7fcc9c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 --2- 172.21.15.110:0/2867821574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc9c100680 0x7fcc9c199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 >> 172.21.15.110:0/2867821574 conn(0x7fcc9c0fbbf0 msgr2=0x7fcc9c068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:04.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 shutdown_connections 2024-04-25T00:00:04.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.529+0000 7fcca3bf9640 1 -- 172.21.15.110:0/2867821574 wait complete. 2024-04-25T00:00:04.541 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T00:00:04.541 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T00:00:04.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 -- 172.21.15.110:0/1466775073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb500ffa50 msgr2=0x7fdb500ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 --2- 172.21.15.110:0/1466775073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb500ffa50 0x7fdb500ffe70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009930 tx=0x7fdb4c02a420 comp rx=0 tx=0).stop 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 -- 172.21.15.110:0/1466775073 shutdown_connections 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 --2- 172.21.15.110:0/1466775073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb501033e0 0x7fdb50109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 --2- 172.21.15.110:0/1466775073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb50102a20 0x7fdb50102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 --2- 172.21.15.110:0/1466775073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb500ffa50 0x7fdb500ffe70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 -- 172.21.15.110:0/1466775073 >> 172.21.15.110:0/1466775073 conn(0x7fdb500fb380 msgr2=0x7fdb500fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 -- 172.21.15.110:0/1466775073 shutdown_connections 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 -- 172.21.15.110:0/1466775073 wait complete. 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.642+0000 7fdb58163640 1 Processor -- start 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 -- start start 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb500ffa50 0x7fdb50102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:04.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb50102a20 0x7fdb501009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb501033e0 0x7fdb50100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010d320 con 0x7fdb501033e0 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010d1a0 con 0x7fdb500ffa50 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010d4a0 con 0x7fdb50102a20 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb501033e0 0x7fdb50100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb501033e0 0x7fdb50100f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58244/0 (socket says 172.21.15.110:58244) 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 -- 172.21.15.110:0/1334164502 learned_addr learned my addr 172.21.15.110:0/1334164502 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb55eff640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb500ffa50 0x7fdb50102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 -- 172.21.15.110:0/1334164502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb50102a20 msgr2=0x7fdb501009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb50102a20 0x7fdb501009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 -- 172.21.15.110:0/1334164502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb500ffa50 msgr2=0x7fdb50102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb500ffa50 0x7fdb50102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 -- 172.21.15.110:0/1334164502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb501017a0 con 0x7fdb501033e0 2024-04-25T00:00:04.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb55eff640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb500ffa50 0x7fdb50102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:04.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb56700640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb501033e0 0x7fdb50100f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800eaa0 tx=0x7fdb4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:04.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb46ffd640 1 -- 172.21.15.110:0/1334164502 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800cdb0 con 0x7fdb501033e0 2024-04-25T00:00:04.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb46ffd640 1 -- 172.21.15.110:0/1334164502 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb48005020 con 0x7fdb501033e0 2024-04-25T00:00:04.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb501a1b60 con 0x7fdb501033e0 2024-04-25T00:00:04.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb46ffd640 1 -- 172.21.15.110:0/1334164502 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480116d0 con 0x7fdb501033e0 2024-04-25T00:00:04.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.643+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb501a2060 con 0x7fdb501033e0 2024-04-25T00:00:04.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.644+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb50108c70 con 0x7fdb501033e0 2024-04-25T00:00:04.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.646+0000 7fdb46ffd640 1 -- 172.21.15.110:0/1334164502 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdb480051d0 con 0x7fdb501033e0 2024-04-25T00:00:04.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.646+0000 7fdb46ffd640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdb3c0a2b90 0x7fdb3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:04.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.647+0000 7fdb55eff640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdb3c0a2b90 0x7fdb3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:04.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.647+0000 7fdb46ffd640 1 -- 172.21.15.110:0/1334164502 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fdb480cf520 con 0x7fdb501033e0 2024-04-25T00:00:04.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.647+0000 7fdb55eff640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdb3c0a2b90 0x7fdb3c0a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009230 tx=0x7fdb4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:04.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.650+0000 7fdb46ffd640 1 -- 172.21.15.110:0/1334164502 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdb48016030 con 0x7fdb501033e0 2024-04-25T00:00:04.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:04 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3814088408' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:00:04.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:04 smithi110 ceph-mon[29187]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:00:04.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:04 smithi110 ceph-mon[29187]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:00:04.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:04 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3058580047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:04.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:04 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/234411857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:04.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:04 smithi110 ceph-mon[29187]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:00:04.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:04 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2867821574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:00:04.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:04 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3814088408' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:00:04.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:04 smithi173 ceph-mon[32402]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:00:04.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:04 smithi173 ceph-mon[32402]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:00:04.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:04 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3058580047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:04.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:04 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/234411857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:04.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:04 smithi173 ceph-mon[32402]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:00:04.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:04 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2867821574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:00:04.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:04.865+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fdb5002d370 con 0x7fdb501033e0 2024-04-25T00:00:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:04 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3814088408' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:00:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:04 smithi188 ceph-mon[33285]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:00:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:04 smithi188 ceph-mon[33285]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:00:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:04 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3058580047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:04 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/234411857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:04 smithi188 ceph-mon[33285]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:00:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:04 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2867821574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:00:05.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.535+0000 7fdb46ffd640 1 -- 172.21.15.110:0/1334164502 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7fdb4809c5d0 con 0x7fdb501033e0 2024-04-25T00:00:05.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdb3c0a2b90 msgr2=0x7fdb3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:05.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdb3c0a2b90 0x7fdb3c0a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009230 tx=0x7fdb4c005fd0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb501033e0 msgr2=0x7fdb50100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb501033e0 0x7fdb50100f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800eaa0 tx=0x7fdb4800ef60 comp rx=0 tx=0).stop 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 shutdown_connections 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fdb3c0a2b90 0x7fdb3c0a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb501033e0 0x7fdb50100f10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb50102a20 0x7fdb501009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 --2- 172.21.15.110:0/1334164502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb500ffa50 0x7fdb50102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 >> 172.21.15.110:0/1334164502 conn(0x7fdb500fb380 msgr2=0x7fdb50105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 shutdown_connections 2024-04-25T00:00:05.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.537+0000 7fdb58163640 1 -- 172.21.15.110:0/1334164502 wait complete. 2024-04-25T00:00:05.552 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 -- 172.21.15.110:0/1229224057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c0ffe20 msgr2=0x7fa97c100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 --2- 172.21.15.110:0/1229224057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c0ffe20 0x7fa97c100240 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa978009900 tx=0x7fa97802a410 comp rx=0 tx=0).stop 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 -- 172.21.15.110:0/1229224057 shutdown_connections 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 --2- 172.21.15.110:0/1229224057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa97c101930 0x7fa97c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 --2- 172.21.15.110:0/1229224057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa97c100f70 0x7fa97c1013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 --2- 172.21.15.110:0/1229224057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c0ffe20 0x7fa97c100240 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 -- 172.21.15.110:0/1229224057 >> 172.21.15.110:0/1229224057 conn(0x7fa97c0fb370 msgr2=0x7fa97c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 -- 172.21.15.110:0/1229224057 shutdown_connections 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 -- 172.21.15.110:0/1229224057 wait complete. 2024-04-25T00:00:05.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 Processor -- start 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.657+0000 7fa984676640 1 -- start start 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa984676640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa97c0ffe20 0x7fa97c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa984676640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa97c100f70 0x7fa97c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa984676640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c101930 0x7fa97c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa984676640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10b9f0 con 0x7fa97c101930 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa984676640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10b870 con 0x7fa97c100f70 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa984676640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c10bb70 con 0x7fa97c0ffe20 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c101930 0x7fa97c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c101930 0x7fa97c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58260/0 (socket says 172.21.15.110:58260) 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 -- 172.21.15.110:0/3328014481 learned_addr learned my addr 172.21.15.110:0/3328014481 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982412640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa97c0ffe20 0x7fa97c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:05.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa981c11640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa97c100f70 0x7fa97c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 -- 172.21.15.110:0/3328014481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa97c0ffe20 msgr2=0x7fa97c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa97c0ffe20 0x7fa97c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 -- 172.21.15.110:0/3328014481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa97c100f70 msgr2=0x7fa97c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa97c100f70 0x7fa97c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 -- 172.21.15.110:0/3328014481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c19f710 con 0x7fa97c101930 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa981c11640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa97c100f70 0x7fa97c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa982c13640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c101930 0x7fa97c19f0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa974009fd0 tx=0x7fa97400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa9737fe640 1 -- 172.21.15.110:0/3328014481 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974017070 con 0x7fa97c101930 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa9737fe640 1 -- 172.21.15.110:0/3328014481 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa974003e20 con 0x7fa97c101930 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa9737fe640 1 -- 172.21.15.110:0/3328014481 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974010460 con 0x7fa97c101930 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c19fa00 con 0x7fa97c101930 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.658+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c19ff00 con 0x7fa97c101930 2024-04-25T00:00:05.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.659+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97c1a0130 con 0x7fa97c101930 2024-04-25T00:00:05.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.660+0000 7fa9737fe640 1 -- 172.21.15.110:0/3328014481 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa974003970 con 0x7fa97c101930 2024-04-25T00:00:05.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.661+0000 7fa9737fe640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa9680a2b90 0x7fa9680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:05.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.661+0000 7fa982412640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa9680a2b90 0x7fa9680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:05.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.661+0000 7fa9737fe640 1 -- 172.21.15.110:0/3328014481 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa9740cee30 con 0x7fa97c101930 2024-04-25T00:00:05.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.661+0000 7fa982412640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa9680a2b90 0x7fa9680a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa978000c00 tx=0x7fa9780314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:05.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.666+0000 7fa9737fe640 1 -- 172.21.15.110:0/3328014481 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa97409bee0 con 0x7fa97c101930 2024-04-25T00:00:05.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:05 smithi110 ceph-mon[29187]: pgmap v290: 97 pgs: 38 creating+peering, 32 unknown, 27 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-25T00:00:05.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:05 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2867821574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:00:05.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:05 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] up:active 2024-04-25T00:00:05.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:05 smithi110 ceph-mon[29187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:00:05.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:05 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1334164502' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:00:05.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:05 smithi110 ceph-mon[29187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:00:05.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:05 smithi110 ceph-mon[29187]: Cluster is now healthy 2024-04-25T00:00:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:05 smithi173 ceph-mon[32402]: pgmap v290: 97 pgs: 38 creating+peering, 32 unknown, 27 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-25T00:00:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:05 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2867821574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:00:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:05 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] up:active 2024-04-25T00:00:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:05 smithi173 ceph-mon[32402]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:00:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:05 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1334164502' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:00:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:05 smithi173 ceph-mon[32402]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:00:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:05 smithi173 ceph-mon[32402]: Cluster is now healthy 2024-04-25T00:00:05.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:05.878+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa97c1a0520 con 0x7fa97c101930 2024-04-25T00:00:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:05 smithi188 ceph-mon[33285]: pgmap v290: 97 pgs: 38 creating+peering, 32 unknown, 27 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-25T00:00:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:05 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2867821574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:00:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:05 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] up:active 2024-04-25T00:00:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:05 smithi188 ceph-mon[33285]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:00:05.920 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:05 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1334164502' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:00:05.920 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:05 smithi188 ceph-mon[33285]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:00:05.920 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:05 smithi188 ceph-mon[33285]: Cluster is now healthy 2024-04-25T00:00:06.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.542+0000 7fa9737fe640 1 -- 172.21.15.110:0/3328014481 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fa97409b7e0 con 0x7fa97c101930 2024-04-25T00:00:06.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa9680a2b90 msgr2=0x7fa9680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:06.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa9680a2b90 0x7fa9680a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa978000c00 tx=0x7fa9780314a0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c101930 msgr2=0x7fa97c19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c101930 0x7fa97c19f0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa974009fd0 tx=0x7fa97400b260 comp rx=0 tx=0).stop 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 shutdown_connections 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa9680a2b90 0x7fa9680a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa97c101930 0x7fa97c19f0a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa97c100f70 0x7fa97c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 --2- 172.21.15.110:0/3328014481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa97c0ffe20 0x7fa97c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 >> 172.21.15.110:0/3328014481 conn(0x7fa97c0fb370 msgr2=0x7fa97c0fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 shutdown_connections 2024-04-25T00:00:06.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.545+0000 7fa984676640 1 -- 172.21.15.110:0/3328014481 wait complete. 2024-04-25T00:00:06.563 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T00:00:06.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 -- 172.21.15.110:0/3782358358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb8102250 msgr2=0x7fabb8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:06.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 --2- 172.21.15.110:0/3782358358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb8102250 0x7fabb8108ae0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ca0 tx=0x7fabb002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 -- 172.21.15.110:0/3782358358 shutdown_connections 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 --2- 172.21.15.110:0/3782358358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb8102250 0x7fabb8108ae0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 --2- 172.21.15.110:0/3782358358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb8101890 0x7fabb8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 --2- 172.21.15.110:0/3782358358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabb81006a0 0x7fabb8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 -- 172.21.15.110:0/3782358358 >> 172.21.15.110:0/3782358358 conn(0x7fabb80fbc10 msgr2=0x7fabb80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 -- 172.21.15.110:0/3782358358 shutdown_connections 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.670+0000 7fabbedc4640 1 -- 172.21.15.110:0/3782358358 wait complete. 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 Processor -- start 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 -- start start 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb81006a0 0x7fabb806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb8101890 0x7fabb819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabb8102250 0x7fabb819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:06.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabb810c1d0 con 0x7fabb81006a0 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fabb810c050 con 0x7fabb8101890 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabb810c350 con 0x7fabb8102250 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb81006a0 0x7fabb806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb81006a0 0x7fabb806a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34404/0 (socket says 172.21.15.110:34404) 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 -- 172.21.15.110:0/1474039157 learned_addr learned my addr 172.21.15.110:0/1474039157 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabb7fff640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb8101890 0x7fabb819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 -- 172.21.15.110:0/1474039157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabb8102250 msgr2=0x7fabb819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabb8102250 0x7fabb819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 -- 172.21.15.110:0/1474039157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb8101890 msgr2=0x7fabb819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb8101890 0x7fabb819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 -- 172.21.15.110:0/1474039157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb819f590 con 0x7fabb81006a0 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbcb60640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb81006a0 0x7fabb806a910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d950 tx=0x7fabac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:06.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabb5ffb640 1 -- 172.21.15.110:0/1474039157 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac017070 con 0x7fabb81006a0 2024-04-25T00:00:06.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb819f880 con 0x7fabb81006a0 2024-04-25T00:00:06.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabb5ffb640 1 -- 172.21.15.110:0/1474039157 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabac009c30 con 0x7fabb81006a0 2024-04-25T00:00:06.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.671+0000 7fabb5ffb640 1 -- 172.21.15.110:0/1474039157 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac00f620 con 0x7fabb81006a0 2024-04-25T00:00:06.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.672+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb819fd80 con 0x7fabb81006a0 2024-04-25T00:00:06.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.673+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb8107a40 con 0x7fabb81006a0 2024-04-25T00:00:06.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.674+0000 7fabb5ffb640 1 -- 172.21.15.110:0/1474039157 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fabac01a070 con 0x7fabb81006a0 2024-04-25T00:00:06.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.674+0000 7fabb5ffb640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faba80a2b90 0x7faba80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:06.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.674+0000 7fabb7fff640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faba80a2b90 0x7faba80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:06.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.674+0000 7fabb5ffb640 1 -- 172.21.15.110:0/1474039157 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fabac0ce950 con 0x7fabb81006a0 2024-04-25T00:00:06.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.675+0000 7fabb7fff640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faba80a2b90 0x7faba80a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabb806a680 tx=0x7faba4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:06.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.678+0000 7fabb5ffb640 1 -- 172.21.15.110:0/1474039157 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fabac009de0 con 0x7fabb81006a0 2024-04-25T00:00:06.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:06 smithi110 ceph-mon[29187]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:00:06.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:06 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1334164502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:00:06.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:06 smithi110 ceph-mon[29187]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:00:06.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:06 smithi110 ceph-mon[29187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:00:06.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:06 smithi110 ceph-mon[29187]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T00:00:06.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:06 smithi110 ceph-mon[29187]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T00:00:06.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:06 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3328014481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:00:06.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:06 smithi173 ceph-mon[32402]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:00:06.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:06 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1334164502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:00:06.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:06 smithi173 ceph-mon[32402]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:00:06.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:06 smithi173 ceph-mon[32402]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:00:06.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:06 smithi173 ceph-mon[32402]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T00:00:06.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:06 smithi173 ceph-mon[32402]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T00:00:06.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:06 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3328014481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:00:06.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:06.896+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fabb802d170 con 0x7fabb81006a0 2024-04-25T00:00:06.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:06 smithi188 ceph-mon[33285]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:00:06.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:06 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1334164502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:00:06.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:06 smithi188 ceph-mon[33285]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:00:06.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:06 smithi188 ceph-mon[33285]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:00:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:06 smithi188 ceph-mon[33285]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T00:00:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:06 smithi188 ceph-mon[33285]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T00:00:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:06 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3328014481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:00:07.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.549+0000 7fabb5ffb640 1 -- 172.21.15.110:0/1474039157 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fabb802d170 con 0x7fabb81006a0 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.551+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faba80a2b90 msgr2=0x7faba80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.551+0000 7fabbedc4640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faba80a2b90 0x7faba80a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabb806a680 tx=0x7faba4005f90 comp rx=0 tx=0).stop 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb81006a0 msgr2=0x7fabb806a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb81006a0 0x7fabb806a910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d950 tx=0x7fabac00de10 comp rx=0 tx=0).stop 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 shutdown_connections 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faba80a2b90 0x7faba80a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabb8102250 0x7fabb819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb8101890 0x7fabb819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 --2- 172.21.15.110:0/1474039157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabb81006a0 0x7fabb806a910 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 >> 172.21.15.110:0/1474039157 conn(0x7fabb80fbc10 msgr2=0x7fabb8104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:07.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 shutdown_connections 2024-04-25T00:00:07.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.552+0000 7fabbedc4640 1 -- 172.21.15.110:0/1474039157 wait complete. 2024-04-25T00:00:07.565 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T00:00:07.566 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T00:00:07.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.669+0000 7f35978a5640 1 -- 172.21.15.110:0/557453496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3590101890 msgr2=0x7f3590101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:07.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.669+0000 7f35978a5640 1 --2- 172.21.15.110:0/557453496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3590101890 0x7f3590101d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3580009900 tx=0x7f358002a410 comp rx=0 tx=0).stop 2024-04-25T00:00:07.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 -- 172.21.15.110:0/557453496 shutdown_connections 2024-04-25T00:00:07.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 --2- 172.21.15.110:0/557453496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3590102250 0x7f3590108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 --2- 172.21.15.110:0/557453496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3590101890 0x7f3590101d10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 --2- 172.21.15.110:0/557453496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35901006a0 0x7f3590100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 -- 172.21.15.110:0/557453496 >> 172.21.15.110:0/557453496 conn(0x7f35900fbc30 msgr2=0x7f35900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:07.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 -- 172.21.15.110:0/557453496 shutdown_connections 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 -- 172.21.15.110:0/557453496 wait complete. 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 Processor -- start 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 -- start start 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35901006a0 0x7f359006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590101890 0x7f359019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3590102250 0x7f359019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f359010c1b0 con 0x7f35901006a0 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f359010c030 con 0x7f3590102250 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f35978a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f359010c330 con 0x7f3590101890 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f3595641640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35901006a0 0x7f359006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f3595641640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35901006a0 0x7f359006a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34432/0 (socket says 172.21.15.110:34432) 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.670+0000 7f3595641640 1 -- 172.21.15.110:0/4027290236 learned_addr learned my addr 172.21.15.110:0/4027290236 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f3594e40640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590101890 0x7f359019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:07.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f3595e42640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3590102250 0x7f359019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f3595641640 1 -- 172.21.15.110:0/4027290236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590101890 msgr2=0x7f359019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f3595641640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590101890 0x7f359019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f3595641640 1 -- 172.21.15.110:0/4027290236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3590102250 msgr2=0x7f359019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f3595641640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3590102250 0x7f359019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f3595641640 1 -- 172.21.15.110:0/4027290236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359019f5f0 con 0x7f35901006a0 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f3595641640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35901006a0 0x7f359006a8d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3588009fd0 tx=0x7f358800c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f35867fc640 1 -- 172.21.15.110:0/4027290236 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588011470 con 0x7f35901006a0 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f35867fc640 1 -- 172.21.15.110:0/4027290236 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3588011ac0 con 0x7f35901006a0 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359019f8e0 con 0x7f35901006a0 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f35867fc640 1 -- 172.21.15.110:0/4027290236 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358801aab0 con 0x7f35901006a0 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.671+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359019fe00 con 0x7f35901006a0 2024-04-25T00:00:07.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.672+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3590107ab0 con 0x7f35901006a0 2024-04-25T00:00:07.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.674+0000 7f35867fc640 1 -- 172.21.15.110:0/4027290236 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3588011c70 con 0x7f35901006a0 2024-04-25T00:00:07.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.674+0000 7f35867fc640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f357c0a2b90 0x7f357c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:07.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.674+0000 7f3594e40640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f357c0a2b90 0x7f357c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:07.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.674+0000 7f35867fc640 1 -- 172.21.15.110:0/4027290236 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f35880cf5b0 con 0x7f35901006a0 2024-04-25T00:00:07.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.675+0000 7f3594e40640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f357c0a2b90 0x7f357c0a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3580006010 tx=0x7f3580000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:07.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.678+0000 7f35867fc640 1 -- 172.21.15.110:0/4027290236 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3588016030 con 0x7f35901006a0 2024-04-25T00:00:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: pgmap v291: 97 pgs: 38 creating+peering, 19 unknown, 40 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 4 op/s 2024-04-25T00:00:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3328014481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:00:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] up:active 2024-04-25T00:00:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:00:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:00:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: Cluster is now healthy 2024-04-25T00:00:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T00:00:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T00:00:07.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T00:00:07.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:07 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1474039157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:00:07.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.879+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3590061b80 con 0x7f357c0a2b90 2024-04-25T00:00:07.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: pgmap v291: 97 pgs: 38 creating+peering, 19 unknown, 40 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 4 op/s 2024-04-25T00:00:07.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3328014481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:00:07.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] up:active 2024-04-25T00:00:07.885 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:00:07.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:00:07.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: Cluster is now healthy 2024-04-25T00:00:07.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T00:00:07.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T00:00:07.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T00:00:07.886 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1474039157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:00:07.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.903+0000 7f35867fc640 1 -- 172.21.15.110:0/4027290236 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3590061b80 con 0x7f357c0a2b90 2024-04-25T00:00:07.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f357c0a2b90 msgr2=0x7f357c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:07.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f357c0a2b90 0x7f357c0a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3580006010 tx=0x7f3580000bc0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35901006a0 msgr2=0x7f359006a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:07.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35901006a0 0x7f359006a8d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3588009fd0 tx=0x7f358800c270 comp rx=0 tx=0).stop 2024-04-25T00:00:07.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 shutdown_connections 2024-04-25T00:00:07.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f357c0a2b90 0x7f357c0a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3590102250 0x7f359019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590101890 0x7f359019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 --2- 172.21.15.110:0/4027290236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35901006a0 0x7f359006a8d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:07.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 >> 172.21.15.110:0/4027290236 conn(0x7f35900fbc30 msgr2=0x7f3590104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:07.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 shutdown_connections 2024-04-25T00:00:07.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:07.905+0000 7f35978a5640 1 -- 172.21.15.110:0/4027290236 wait complete. 2024-04-25T00:00:07.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: pgmap v291: 97 pgs: 38 creating+peering, 19 unknown, 40 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s wr, 4 op/s 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3328014481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] up:active 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: Cluster is now healthy 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T00:00:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T00:00:08.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:07 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1474039157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:00:08.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 -- 172.21.15.110:0/3921676885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7824102d90 msgr2=0x7f78241031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 --2- 172.21.15.110:0/3921676885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7824102d90 0x7f78241031b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7814009900 tx=0x7f781402a410 comp rx=0 tx=0).stop 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 -- 172.21.15.110:0/3921676885 shutdown_connections 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 --2- 172.21.15.110:0/3921676885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78241001c0 0x7f7824109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 --2- 172.21.15.110:0/3921676885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78240ff800 0x7f78240ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 --2- 172.21.15.110:0/3921676885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7824102d90 0x7f78241031b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 -- 172.21.15.110:0/3921676885 >> 172.21.15.110:0/3921676885 conn(0x7f78240fb380 msgr2=0x7f78240fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 -- 172.21.15.110:0/3921676885 shutdown_connections 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.019+0000 7f7828ccd640 1 -- 172.21.15.110:0/3921676885 wait complete. 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 Processor -- start 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 -- start start 2024-04-25T00:00:08.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78240ff800 0x7f7824197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78241001c0 0x7f7824197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824102d90 0x7f782419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f782410d2c0 con 0x7f78240ff800 2024-04-25T00:00:08.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f782410d140 con 0x7f7824102d90 2024-04-25T00:00:08.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f782410d440 con 0x7f78241001c0 2024-04-25T00:00:08.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78240ff800 0x7f7824197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78240ff800 0x7f7824197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34446/0 (socket says 172.21.15.110:34446) 2024-04-25T00:00:08.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 -- 172.21.15.110:0/1983374509 learned_addr learned my addr 172.21.15.110:0/1983374509 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:08.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7821d9b640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78241001c0 0x7f7824197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7822d9d640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824102d90 0x7f782419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 -- 172.21.15.110:0/1983374509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78241001c0 msgr2=0x7f7824197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78241001c0 0x7f7824197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 -- 172.21.15.110:0/1983374509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824102d90 msgr2=0x7f782419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824102d90 0x7f782419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 -- 172.21.15.110:0/1983374509 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782419f660 con 0x7f78240ff800 2024-04-25T00:00:08.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f782259c640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78240ff800 0x7f7824197a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f78140314e0 tx=0x7f7814031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:08.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f78137fe640 1 -- 172.21.15.110:0/1983374509 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781403f030 con 0x7f78240ff800 2024-04-25T00:00:08.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f78137fe640 1 -- 172.21.15.110:0/1983374509 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78140044b0 con 0x7f78240ff800 2024-04-25T00:00:08.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782419f8f0 con 0x7f78240ff800 2024-04-25T00:00:08.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.020+0000 7f78137fe640 1 -- 172.21.15.110:0/1983374509 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814035070 con 0x7f78240ff800 2024-04-25T00:00:08.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.021+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782419fde0 con 0x7f78240ff800 2024-04-25T00:00:08.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.021+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7824108d70 con 0x7f78240ff800 2024-04-25T00:00:08.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.022+0000 7f78137fe640 1 -- 172.21.15.110:0/1983374509 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7814042030 con 0x7f78240ff800 2024-04-25T00:00:08.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.023+0000 7f78137fe640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f780c0a2b90 0x7f780c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.023+0000 7f7821d9b640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f780c0a2b90 0x7f780c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.023+0000 7f78137fe640 1 -- 172.21.15.110:0/1983374509 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f78140ef3f0 con 0x7f78240ff800 2024-04-25T00:00:08.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.023+0000 7f7821d9b640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f780c0a2b90 0x7f780c0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7824198f50 tx=0x7f781800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:08.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.027+0000 7f78137fe640 1 -- 172.21.15.110:0/1983374509 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f78140bbe50 con 0x7f78240ff800 2024-04-25T00:00:08.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.234+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7824061c10 con 0x7f780c0a2b90 2024-04-25T00:00:08.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.257+0000 7f78137fe640 1 -- 172.21.15.110:0/1983374509 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7824061c10 con 0x7f780c0a2b90 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f780c0a2b90 msgr2=0x7f780c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f780c0a2b90 0x7f780c0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7824198f50 tx=0x7f781800b390 comp rx=0 tx=0).stop 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78240ff800 msgr2=0x7f7824197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78240ff800 0x7f7824197a40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f78140314e0 tx=0x7f7814031a30 comp rx=0 tx=0).stop 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 shutdown_connections 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f780c0a2b90 0x7f780c0a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7824102d90 0x7f782419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78241001c0 0x7f7824197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 --2- 172.21.15.110:0/1983374509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78240ff800 0x7f7824197a40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 >> 172.21.15.110:0/1983374509 conn(0x7f78240fb380 msgr2=0x7f7824105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.259+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 shutdown_connections 2024-04-25T00:00:08.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.260+0000 7f7828ccd640 1 -- 172.21.15.110:0/1983374509 wait complete. 2024-04-25T00:00:08.270 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-25T00:00:07.884+0000 7f51470b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:00:08.270 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-25T00:00:07.884+0000 7f51470b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:00:08.270 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-25T00:00:07.884+0000 7f51470b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:00:08.270 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-25T00:00:07.884+0000 7f51470b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:00:08.270 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:00:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-25T00:00:07.884+0000 7f51470b3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:00:08.271 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T00:00:08.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 -- 172.21.15.110:0/1576164845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc104b30 msgr2=0x7f8bfc106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 --2- 172.21.15.110:0/1576164845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc104b30 0x7f8bfc106f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009900 tx=0x7f8bf002a410 comp rx=0 tx=0).stop 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 -- 172.21.15.110:0/1576164845 shutdown_connections 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 --2- 172.21.15.110:0/1576164845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc107460 0x7f8bfc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 --2- 172.21.15.110:0/1576164845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc104b30 0x7f8bfc106f20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 --2- 172.21.15.110:0/1576164845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bfc102200 0x7f8bfc1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 -- 172.21.15.110:0/1576164845 >> 172.21.15.110:0/1576164845 conn(0x7f8bfc0fbbf0 msgr2=0x7f8bfc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 -- 172.21.15.110:0/1576164845 shutdown_connections 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.374+0000 7f8c02570640 1 -- 172.21.15.110:0/1576164845 wait complete. 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c02570640 1 Processor -- start 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c02570640 1 -- start start 2024-04-25T00:00:08.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c02570640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc102200 0x7f8bfc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c02570640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bfc104b30 0x7f8bfc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c02570640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc107460 0x7f8bfc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c02570640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10d300 con 0x7f8bfc102200 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c02570640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10d180 con 0x7f8bfc104b30 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c02570640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10d480 con 0x7f8bfc107460 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc102200 0x7f8bfc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc102200 0x7f8bfc197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34466/0 (socket says 172.21.15.110:34466) 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8bfbfff640 1 -- 172.21.15.110:0/2110238530 learned_addr learned my addr 172.21.15.110:0/2110238530 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8bfb7fe640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bfc104b30 0x7f8bfc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8c00b0d640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc107460 0x7f8bfc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.375+0000 7f8bfbfff640 1 -- 172.21.15.110:0/2110238530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc107460 msgr2=0x7f8bfc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8bfbfff640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc107460 0x7f8bfc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8bfbfff640 1 -- 172.21.15.110:0/2110238530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bfc104b30 msgr2=0x7f8bfc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8bfbfff640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bfc104b30 0x7f8bfc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8bfbfff640 1 -- 172.21.15.110:0/2110238530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc19f680 con 0x7f8bfc102200 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8c00b0d640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc107460 0x7f8bfc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8bfbfff640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc102200 0x7f8bfc197a60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00e9f0 tx=0x7f8bec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8bf97fa640 1 -- 172.21.15.110:0/2110238530 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00cd80 con 0x7f8bfc102200 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8bf97fa640 1 -- 172.21.15.110:0/2110238530 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bec0108e0 con 0x7f8bfc102200 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc19f970 con 0x7f8bfc102200 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8bf97fa640 1 -- 172.21.15.110:0/2110238530 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec018920 con 0x7f8bfc102200 2024-04-25T00:00:08.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.376+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc19fe90 con 0x7f8bfc102200 2024-04-25T00:00:08.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.377+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bfc0fd6e0 con 0x7f8bfc102200 2024-04-25T00:00:08.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.378+0000 7f8bf97fa640 1 -- 172.21.15.110:0/2110238530 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8bec010a90 con 0x7f8bfc102200 2024-04-25T00:00:08.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.378+0000 7f8bf97fa640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be40a2b90 0x7f8be40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.379+0000 7f8bfb7fe640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be40a2b90 0x7f8be40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.379+0000 7f8bf97fa640 1 -- 172.21.15.110:0/2110238530 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8bec0cf310 con 0x7f8bfc102200 2024-04-25T00:00:08.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.379+0000 7f8bfb7fe640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be40a2b90 0x7f8be40a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005ab0 tx=0x7f8bf0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:08.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.383+0000 7f8bf97fa640 1 -- 172.21.15.110:0/2110238530 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8bec09c320 con 0x7f8bfc102200 2024-04-25T00:00:08.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.590+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f8bfc061c10 con 0x7f8be40a2b90 2024-04-25T00:00:08.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.595+0000 7f8bf97fa640 1 -- 172.21.15.110:0/2110238530 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8bfc061c10 con 0x7f8be40a2b90 2024-04-25T00:00:08.595 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/qa/sv_0/a6bd4ddc-e93b-4dad-bcd9-0ea4116814e4 2024-04-25T00:00:08.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be40a2b90 msgr2=0x7f8be40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be40a2b90 0x7f8be40a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005ab0 tx=0x7f8bf0005a20 comp rx=0 tx=0).stop 2024-04-25T00:00:08.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc102200 msgr2=0x7f8bfc197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc102200 0x7f8bfc197a60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00e9f0 tx=0x7f8bec00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 shutdown_connections 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8be40a2b90 0x7f8be40a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc107460 0x7f8bfc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bfc104b30 0x7f8bfc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 --2- 172.21.15.110:0/2110238530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bfc102200 0x7f8bfc197a60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 >> 172.21.15.110:0/2110238530 conn(0x7f8bfc0fbbf0 msgr2=0x7f8bfc104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 shutdown_connections 2024-04-25T00:00:08.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.597+0000 7f8c02570640 1 -- 172.21.15.110:0/2110238530 wait complete. 2024-04-25T00:00:08.608 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T00:00:08.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 -- 172.21.15.110:0/135572640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 msgr2=0x7fa1300ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 --2- 172.21.15.110:0/135572640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 0x7fa1300ffca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa120009900 tx=0x7fa12002a410 comp rx=0 tx=0).stop 2024-04-25T00:00:08.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 -- 172.21.15.110:0/135572640 shutdown_connections 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 --2- 172.21.15.110:0/135572640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1301001e0 0x7fa130109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 --2- 172.21.15.110:0/135572640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 0x7fa1300ffca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 --2- 172.21.15.110:0/135572640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa130102db0 0x7fa1301031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 -- 172.21.15.110:0/135572640 >> 172.21.15.110:0/135572640 conn(0x7fa1300fb380 msgr2=0x7fa1300fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 -- 172.21.15.110:0/135572640 shutdown_connections 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.711+0000 7fa1380da640 1 -- 172.21.15.110:0/135572640 wait complete. 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa1380da640 1 Processor -- start 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa1380da640 1 -- start start 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa1380da640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 0x7fa130197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa1380da640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1301001e0 0x7fa130197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa1380da640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa130102db0 0x7fa13019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa1380da640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa13010d2a0 con 0x7fa1300ff820 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa1380da640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa13010d120 con 0x7fa130102db0 2024-04-25T00:00:08.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa1380da640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa13010d420 con 0x7fa1301001e0 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 0x7fa130197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 0x7fa130197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34492/0 (socket says 172.21.15.110:34492) 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 -- 172.21.15.110:0/3805293852 learned_addr learned my addr 172.21.15.110:0/3805293852 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135675640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1301001e0 0x7fa130197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa136677640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa130102db0 0x7fa13019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 -- 172.21.15.110:0/3805293852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1301001e0 msgr2=0x7fa130197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1301001e0 0x7fa130197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 -- 172.21.15.110:0/3805293852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa130102db0 msgr2=0x7fa13019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa130102db0 0x7fa13019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 -- 172.21.15.110:0/3805293852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13019f640 con 0x7fa1300ff820 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa135e76640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 0x7fa130197a20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00d950 tx=0x7fa12c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.712+0000 7fa126ffd640 1 -- 172.21.15.110:0/3805293852 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c017020 con 0x7fa1300ff820 2024-04-25T00:00:08.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.713+0000 7fa126ffd640 1 -- 172.21.15.110:0/3805293852 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa12c009c30 con 0x7fa1300ff820 2024-04-25T00:00:08.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.713+0000 7fa126ffd640 1 -- 172.21.15.110:0/3805293852 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c00f6b0 con 0x7fa1300ff820 2024-04-25T00:00:08.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.713+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13019f930 con 0x7fa1300ff820 2024-04-25T00:00:08.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.713+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13019fe00 con 0x7fa1300ff820 2024-04-25T00:00:08.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.714+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1300689b0 con 0x7fa1300ff820 2024-04-25T00:00:08.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.715+0000 7fa126ffd640 1 -- 172.21.15.110:0/3805293852 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa12c01a020 con 0x7fa1300ff820 2024-04-25T00:00:08.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.715+0000 7fa126ffd640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa11c0a2c10 0x7fa11c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:08.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.715+0000 7fa135675640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa11c0a2c10 0x7fa11c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:08.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.715+0000 7fa126ffd640 1 -- 172.21.15.110:0/3805293852 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa12c0cea00 con 0x7fa1300ff820 2024-04-25T00:00:08.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.716+0000 7fa135675640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa11c0a2c10 0x7fa11c0a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa120006010 tx=0x7fa120000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:08.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.720+0000 7fa126ffd640 1 -- 172.21.15.110:0/3805293852 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa12c009de0 con 0x7fa1300ff820 2024-04-25T00:00:08.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:08 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1474039157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:00:08.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:08 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] up:active 2024-04-25T00:00:08.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:08 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T00:00:08.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:08 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:00:08.853 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:00:08 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: 2024-04-25T00:00:08.577+0000 7f3c66517700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:00:08.853 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:00:08 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: 2024-04-25T00:00:08.577+0000 7f3c66517700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T00:00:08.853 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:00:08 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: 2024-04-25T00:00:08.577+0000 7f3c66517700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T00:00:08.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.925+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa130061b80 con 0x7fa11c0a2c10 2024-04-25T00:00:08.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.949+0000 7fa126ffd640 1 -- 172.21.15.110:0/3805293852 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa130061b80 con 0x7fa11c0a2c10 2024-04-25T00:00:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.951+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa11c0a2c10 msgr2=0x7fa11c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.951+0000 7fa1380da640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa11c0a2c10 0x7fa11c0a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa120006010 tx=0x7fa120000bc0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.951+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 msgr2=0x7fa130197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.951+0000 7fa1380da640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 0x7fa130197a20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00d950 tx=0x7fa12c00de10 comp rx=0 tx=0).stop 2024-04-25T00:00:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.951+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 shutdown_connections 2024-04-25T00:00:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.952+0000 7fa1380da640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa11c0a2c10 0x7fa11c0a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.952+0000 7fa1380da640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa130102db0 0x7fa13019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.952+0000 7fa1380da640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1301001e0 0x7fa130197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.952+0000 7fa1380da640 1 --2- 172.21.15.110:0/3805293852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1300ff820 0x7fa130197a20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:08.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.952+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 >> 172.21.15.110:0/3805293852 conn(0x7fa1300fb380 msgr2=0x7fa130105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:08.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.952+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 shutdown_connections 2024-04-25T00:00:08.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:08.952+0000 7fa1380da640 1 -- 172.21.15.110:0/3805293852 wait complete. 2024-04-25T00:00:08.961 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:08 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1474039157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:00:08.962 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:08 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] up:active 2024-04-25T00:00:08.962 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:08 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T00:00:08.962 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:08 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:00:08.963 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T00:00:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:08 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1474039157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:00:09.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:08 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] up:active 2024-04-25T00:00:09.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:08 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T00:00:09.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:08 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 -- 172.21.15.110:0/2199085414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 msgr2=0x7f0674100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 --2- 172.21.15.110:0/2199085414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 0x7f0674100aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0664009ca0 tx=0x7f066402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 -- 172.21.15.110:0/2199085414 shutdown_connections 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 --2- 172.21.15.110:0/2199085414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0674102230 0x7f0674108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 --2- 172.21.15.110:0/2199085414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0674101870 0x7f0674101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 --2- 172.21.15.110:0/2199085414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 0x7f0674100aa0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 -- 172.21.15.110:0/2199085414 >> 172.21.15.110:0/2199085414 conn(0x7f06740fbbf0 msgr2=0x7f06740fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 -- 172.21.15.110:0/2199085414 shutdown_connections 2024-04-25T00:00:09.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.065+0000 7f067915a640 1 -- 172.21.15.110:0/2199085414 wait complete. 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.066+0000 7f067915a640 1 Processor -- start 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.066+0000 7f067915a640 1 -- start start 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.066+0000 7f067915a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 0x7f0674197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.066+0000 7f067915a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0674101870 0x7f0674197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.066+0000 7f067915a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0674102230 0x7f067419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.066+0000 7f067915a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f067410c200 con 0x7f0674100680 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.066+0000 7f067915a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f067410c080 con 0x7f0674101870 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.066+0000 7f067915a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f067410c380 con 0x7f0674102230 2024-04-25T00:00:09.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 0x7f0674197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 0x7f0674197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34512/0 (socket says 172.21.15.110:34512) 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 -- 172.21.15.110:0/1053621582 learned_addr learned my addr 172.21.15.110:0/1053621582 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f067359e640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0674102230 0x7f067419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f067259c640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0674101870 0x7f0674197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 -- 172.21.15.110:0/1053621582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0674102230 msgr2=0x7f067419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0674102230 0x7f067419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 -- 172.21.15.110:0/1053621582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0674101870 msgr2=0x7f0674197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0674101870 0x7f0674197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 -- 172.21.15.110:0/1053621582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067419f650 con 0x7f0674100680 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f067259c640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0674101870 0x7f0674197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:09.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f0672d9d640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 0x7f0674197a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0664003ba0 tx=0x7f0664033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f065bfff640 1 -- 172.21.15.110:0/1053621582 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664004040 con 0x7f0674100680 2024-04-25T00:00:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067419f8e0 con 0x7f0674100680 2024-04-25T00:00:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f065bfff640 1 -- 172.21.15.110:0/1053621582 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0664004690 con 0x7f0674100680 2024-04-25T00:00:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f065bfff640 1 -- 172.21.15.110:0/1053621582 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066402f4c0 con 0x7f0674100680 2024-04-25T00:00:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.067+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067419fe30 con 0x7f0674100680 2024-04-25T00:00:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.068+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06741a0060 con 0x7f0674100680 2024-04-25T00:00:09.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.069+0000 7f065bfff640 1 -- 172.21.15.110:0/1053621582 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0664030650 con 0x7f0674100680 2024-04-25T00:00:09.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.070+0000 7f065bfff640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f065c0a2b90 0x7f065c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.070+0000 7f067259c640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f065c0a2b90 0x7f065c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.070+0000 7f065bfff640 1 -- 172.21.15.110:0/1053621582 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f06640f09c0 con 0x7f0674100680 2024-04-25T00:00:09.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.070+0000 7f067259c640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f065c0a2b90 0x7f065c0a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0668006fd0 tx=0x7f0668008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:09.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.074+0000 7f065bfff640 1 -- 172.21.15.110:0/1053621582 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f06640f2e20 con 0x7f0674100680 2024-04-25T00:00:09.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.282+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0674068fe0 con 0x7f065c0a2b90 2024-04-25T00:00:09.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.287+0000 7f065bfff640 1 -- 172.21.15.110:0/1053621582 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0674068fe0 con 0x7f065c0a2b90 2024-04-25T00:00:09.287 INFO:teuthology.orchestra.run.smithi110.stdout:/volumes/qa/sv_1/bb9e9b88-c5ee-4348-8493-407b939321c7 2024-04-25T00:00:09.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f065c0a2b90 msgr2=0x7f065c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f065c0a2b90 0x7f065c0a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0668006fd0 tx=0x7f0668008040 comp rx=0 tx=0).stop 2024-04-25T00:00:09.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 msgr2=0x7f0674197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.289 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 0x7f0674197a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0664003ba0 tx=0x7f0664033000 comp rx=0 tx=0).stop 2024-04-25T00:00:09.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 shutdown_connections 2024-04-25T00:00:09.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f065c0a2b90 0x7f065c0a5050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0674102230 0x7f067419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0674101870 0x7f0674197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 --2- 172.21.15.110:0/1053621582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0674100680 0x7f0674197a30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 >> 172.21.15.110:0/1053621582 conn(0x7f06740fbbf0 msgr2=0x7f06740688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:09.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 shutdown_connections 2024-04-25T00:00:09.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.289+0000 7f067915a640 1 -- 172.21.15.110:0/1053621582 wait complete. 2024-04-25T00:00:09.300 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:00:09.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 -- 172.21.15.110:0/664993551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 msgr2=0x7f4b481045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/664993551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 0x7f4b481045f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009900 tx=0x7f4b3802a410 comp rx=0 tx=0).stop 2024-04-25T00:00:09.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 -- 172.21.15.110:0/664993551 shutdown_connections 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/664993551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48107460 0x7f4b481098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/664993551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48104b30 0x7f4b48106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/664993551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 0x7f4b481045f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 -- 172.21.15.110:0/664993551 >> 172.21.15.110:0/664993551 conn(0x7f4b480fbbf0 msgr2=0x7f4b480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 -- 172.21.15.110:0/664993551 shutdown_connections 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.398+0000 7f4b4dd90640 1 -- 172.21.15.110:0/664993551 wait complete. 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b4dd90640 1 Processor -- start 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b4dd90640 1 -- start start 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b4dd90640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 0x7f4b48197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b4dd90640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48104b30 0x7f4b48197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b4dd90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48107460 0x7f4b4819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b4dd90640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d320 con 0x7f4b48102200 2024-04-25T00:00:09.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b4dd90640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d1a0 con 0x7f4b48104b30 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b4dd90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d4a0 con 0x7f4b48107460 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 0x7f4b48197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 0x7f4b48197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34540/0 (socket says 172.21.15.110:34540) 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b477fe640 1 -- 172.21.15.110:0/2349687054 learned_addr learned my addr 172.21.15.110:0/2349687054 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.399+0000 7f4b3effd640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48104b30 0x7f4b48197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b477fe640 1 -- 172.21.15.110:0/2349687054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48107460 msgr2=0x7f4b4819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b477fe640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48107460 0x7f4b4819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b477fe640 1 -- 172.21.15.110:0/2349687054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48104b30 msgr2=0x7f4b48197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b477fe640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48104b30 0x7f4b48197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b477fe640 1 -- 172.21.15.110:0/2349687054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4819f670 con 0x7f4b48102200 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b477fe640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 0x7f4b48197a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005940 tx=0x7f4b38004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b457fa640 1 -- 172.21.15.110:0/2349687054 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3803f030 con 0x7f4b48102200 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b457fa640 1 -- 172.21.15.110:0/2349687054 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b38003cb0 con 0x7f4b48102200 2024-04-25T00:00:09.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4819f900 con 0x7f4b48102200 2024-04-25T00:00:09.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b457fa640 1 -- 172.21.15.110:0/2349687054 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38035070 con 0x7f4b48102200 2024-04-25T00:00:09.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.400+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4819fe70 con 0x7f4b48102200 2024-04-25T00:00:09.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.401+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b480fd6e0 con 0x7f4b48102200 2024-04-25T00:00:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.402+0000 7f4b457fa640 1 -- 172.21.15.110:0/2349687054 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4b38042030 con 0x7f4b48102200 2024-04-25T00:00:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.402+0000 7f4b457fa640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b340a2b90 0x7f4b340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.403+0000 7f4b3effd640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b340a2b90 0x7f4b340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.403+0000 7f4b457fa640 1 -- 172.21.15.110:0/2349687054 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4b380ef9b0 con 0x7f4b48102200 2024-04-25T00:00:09.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.403+0000 7f4b3effd640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b340a2b90 0x7f4b340a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b48198f60 tx=0x7f4b3000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:09.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.407+0000 7f4b457fa640 1 -- 172.21.15.110:0/2349687054 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b380bca40 con 0x7f4b48102200 2024-04-25T00:00:09.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.568+0000 7f4b457fa640 1 -- 172.21.15.110:0/2349687054 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4b380b89b0 con 0x7f4b48102200 2024-04-25T00:00:09.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.625+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b4802d340 con 0x7f4b48102200 2024-04-25T00:00:09.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.626+0000 7f4b457fa640 1 -- 172.21.15.110:0/2349687054 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7f4b380e4eb0 con 0x7f4b48102200 2024-04-25T00:00:09.630 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:09.630 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:00:02.856564+0000","modified":"2024-04-25T00:00:07.553801+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24457,24476,24512]},"id":1}]} 2024-04-25T00:00:09.631 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b340a2b90 msgr2=0x7f4b340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b340a2b90 0x7f4b340a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b48198f60 tx=0x7f4b3000a2d0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 msgr2=0x7f4b48197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 0x7f4b48197a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005940 tx=0x7f4b38004430 comp rx=0 tx=0).stop 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 shutdown_connections 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4b340a2b90 0x7f4b340a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48107460 0x7f4b4819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48104b30 0x7f4b48197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 --2- 172.21.15.110:0/2349687054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b48102200 0x7f4b48197a50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 >> 172.21.15.110:0/2349687054 conn(0x7f4b480fbbf0 msgr2=0x7f4b48102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 shutdown_connections 2024-04-25T00:00:09.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.628+0000 7f4b4dd90640 1 -- 172.21.15.110:0/2349687054 wait complete. 2024-04-25T00:00:09.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:00:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.736+0000 7fc1f1876640 1 -- 172.21.15.110:0/2539137382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 msgr2=0x7fc1ec108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.736+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2539137382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 0x7fc1ec108ad0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009ca0 tx=0x7fc1e402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.736+0000 7fc1f1876640 1 -- 172.21.15.110:0/2539137382 shutdown_connections 2024-04-25T00:00:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.736+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2539137382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 0x7fc1ec108ad0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.736+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2539137382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ec101880 0x7fc1ec101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.736+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2539137382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec100690 0x7fc1ec100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.736+0000 7fc1f1876640 1 -- 172.21.15.110:0/2539137382 >> 172.21.15.110:0/2539137382 conn(0x7fc1ec0fbc20 msgr2=0x7fc1ec0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.736+0000 7fc1f1876640 1 -- 172.21.15.110:0/2539137382 shutdown_connections 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 -- 172.21.15.110:0/2539137382 wait complete. 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 Processor -- start 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 -- start start 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ec100690 0x7fc1ec197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec101880 0x7fc1ec197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 0x7fc1ec19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec10d2e0 con 0x7fc1ec102240 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec10d160 con 0x7fc1ec100690 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1f1876640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec10d460 con 0x7fc1ec101880 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1eb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 0x7fc1ec19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1eb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 0x7fc1ec19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34554/0 (socket says 172.21.15.110:34554) 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.737+0000 7fc1eb7fe640 1 -- 172.21.15.110:0/2388758852 learned_addr learned my addr 172.21.15.110:0/2388758852 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:09.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1ea7fc640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec101880 0x7fc1ec197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1eaffd640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ec100690 0x7fc1ec197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1eb7fe640 1 -- 172.21.15.110:0/2388758852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec101880 msgr2=0x7fc1ec197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1eb7fe640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec101880 0x7fc1ec197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1eb7fe640 1 -- 172.21.15.110:0/2388758852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ec100690 msgr2=0x7fc1ec197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1eb7fe640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ec100690 0x7fc1ec197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1eb7fe640 1 -- 172.21.15.110:0/2388758852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec19f590 con 0x7fc1ec102240 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1eb7fe640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 0x7fc1ec19ef20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009dc0 tx=0x7fc1e4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1f0874640 1 -- 172.21.15.110:0/2388758852 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4032ec0 con 0x7fc1ec102240 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1f0874640 1 -- 172.21.15.110:0/2388758852 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1e40044f0 con 0x7fc1ec102240 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ec19f820 con 0x7fc1ec102240 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1f0874640 1 -- 172.21.15.110:0/2388758852 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4030d40 con 0x7fc1ec102240 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.738+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ec19fd70 con 0x7fc1ec102240 2024-04-25T00:00:09.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.739+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ec1079b0 con 0x7fc1ec102240 2024-04-25T00:00:09.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.740+0000 7fc1f0874640 1 -- 172.21.15.110:0/2388758852 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc1e4004040 con 0x7fc1ec102240 2024-04-25T00:00:09.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.740+0000 7fc1f0874640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc1d40a2dc0 0x7fc1d40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:09.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.741+0000 7fc1eaffd640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc1d40a2dc0 0x7fc1d40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:09.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.741+0000 7fc1f0874640 1 -- 172.21.15.110:0/2388758852 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc1e40f0440 con 0x7fc1ec102240 2024-04-25T00:00:09.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.741+0000 7fc1eaffd640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc1d40a2dc0 0x7fc1d40a5280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec1016e0 tx=0x7fc1e000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.745+0000 7fc1f0874640 1 -- 172.21.15.110:0/2388758852 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc1e40bd500 con 0x7fc1ec102240 2024-04-25T00:00:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:09 smithi173 ceph-mon[32402]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:09 smithi173 ceph-mon[32402]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:09 smithi173 ceph-mon[32402]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.8 KiB/s wr, 17 op/s 2024-04-25T00:00:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:09 smithi173 ceph-mon[32402]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:09.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.921+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1ec02d340 con 0x7fc1ec102240 2024-04-25T00:00:09.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.922+0000 7fc1f0874640 1 -- 172.21.15.110:0/2388758852 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22267 (secure 0 0 0) 0x7fc1e40bce00 con 0x7fc1ec102240 2024-04-25T00:00:09.922 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:09.923 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":85,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:04.124840+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc1d40a2dc0 msgr2=0x7fc1d40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc1d40a2dc0 0x7fc1d40a5280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec1016e0 tx=0x7fc1e000b390 comp rx=0 tx=0).stop 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 msgr2=0x7fc1ec19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 0x7fc1ec19ef20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009dc0 tx=0x7fc1e4003ba0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 shutdown_connections 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc1d40a2dc0 0x7fc1d40a5280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1ec102240 0x7fc1ec19ef20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec101880 0x7fc1ec197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 --2- 172.21.15.110:0/2388758852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ec100690 0x7fc1ec197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 >> 172.21.15.110:0/2388758852 conn(0x7fc1ec0fbc20 msgr2=0x7fc1ec104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 shutdown_connections 2024-04-25T00:00:09.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:09.924+0000 7fc1f1876640 1 -- 172.21.15.110:0/2388758852 wait complete. 2024-04-25T00:00:09.934 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:09 smithi110 ceph-mon[29187]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:09.934 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:09 smithi110 ceph-mon[29187]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:09.934 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:09 smithi110 ceph-mon[29187]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.8 KiB/s wr, 17 op/s 2024-04-25T00:00:09.934 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:09 smithi110 ceph-mon[29187]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:09.936 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:00:09.936 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:00:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:09 smithi188 ceph-mon[33285]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:09 smithi188 ceph-mon[33285]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:09 smithi188 ceph-mon[33285]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.8 KiB/s wr, 17 op/s 2024-04-25T00:00:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:09 smithi188 ceph-mon[33285]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 -- 172.21.15.110:0/2393078975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 msgr2=0x7efe20100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 --2- 172.21.15.110:0/2393078975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 0x7efe20100ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe10009900 tx=0x7efe1002a410 comp rx=0 tx=0).stop 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 -- 172.21.15.110:0/2393078975 shutdown_connections 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 --2- 172.21.15.110:0/2393078975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe20102250 0x7efe20108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 --2- 172.21.15.110:0/2393078975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe20101890 0x7efe20101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 --2- 172.21.15.110:0/2393078975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 0x7efe20100ac0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 -- 172.21.15.110:0/2393078975 >> 172.21.15.110:0/2393078975 conn(0x7efe200fbc30 msgr2=0x7efe200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 -- 172.21.15.110:0/2393078975 shutdown_connections 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 -- 172.21.15.110:0/2393078975 wait complete. 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 Processor -- start 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 -- start start 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 0x7efe20197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:10.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.034+0000 7efe26676640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe20101890 0x7efe20197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe26676640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe20102250 0x7efe2019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe26676640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010c1a0 con 0x7efe201006a0 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe26676640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010c020 con 0x7efe20102250 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe26676640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010c320 con 0x7efe20101890 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 0x7efe20197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 0x7efe20197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34582/0 (socket says 172.21.15.110:34582) 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 -- 172.21.15.110:0/1452169041 learned_addr learned my addr 172.21.15.110:0/1452169041 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 -- 172.21.15.110:0/1452169041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe20101890 msgr2=0x7efe20197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1f7fe640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe20101890 0x7efe20197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe20101890 0x7efe20197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 -- 172.21.15.110:0/1452169041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe20102250 msgr2=0x7efe2019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe24c13640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe20102250 0x7efe2019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe20102250 0x7efe2019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 -- 172.21.15.110:0/1452169041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2019f5b0 con 0x7efe201006a0 2024-04-25T00:00:10.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1f7fe640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe20101890 0x7efe20197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:10.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1ffff640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 0x7efe20197990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efe100314e0 tx=0x7efe100042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:10.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1d7fa640 1 -- 172.21.15.110:0/1452169041 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1003f030 con 0x7efe201006a0 2024-04-25T00:00:10.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2019f840 con 0x7efe201006a0 2024-04-25T00:00:10.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1d7fa640 1 -- 172.21.15.110:0/1452169041 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe10003cb0 con 0x7efe201006a0 2024-04-25T00:00:10.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe1d7fa640 1 -- 172.21.15.110:0/1452169041 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10035070 con 0x7efe201006a0 2024-04-25T00:00:10.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.035+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2019fd30 con 0x7efe201006a0 2024-04-25T00:00:10.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.036+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2019ff90 con 0x7efe201006a0 2024-04-25T00:00:10.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.037+0000 7efe1d7fa640 1 -- 172.21.15.110:0/1452169041 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efe10042030 con 0x7efe201006a0 2024-04-25T00:00:10.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.038+0000 7efe1d7fa640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efe080a2c20 0x7efe080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:10.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.038+0000 7efe1f7fe640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efe080a2c20 0x7efe080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:10.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.038+0000 7efe1d7fa640 1 -- 172.21.15.110:0/1452169041 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7efe100ef7f0 con 0x7efe201006a0 2024-04-25T00:00:10.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.038+0000 7efe1f7fe640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efe080a2c20 0x7efe080a50e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efe20198ea0 tx=0x7efe14005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:10.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.042+0000 7efe1d7fa640 1 -- 172.21.15.110:0/1452169041 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe100bc6a0 con 0x7efe201006a0 2024-04-25T00:00:10.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.213+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe200fde10 con 0x7efe080a2c20 2024-04-25T00:00:10.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.221+0000 7efe1d7fa640 1 -- 172.21.15.110:0/1452169041 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7efe200fde10 con 0x7efe080a2c20 2024-04-25T00:00:10.225 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:10.226 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efe080a2c20 msgr2=0x7efe080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efe080a2c20 0x7efe080a50e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efe20198ea0 tx=0x7efe14005d90 comp rx=0 tx=0).stop 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 msgr2=0x7efe20197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 0x7efe20197990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efe100314e0 tx=0x7efe100042e0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 shutdown_connections 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efe080a2c20 0x7efe080a50e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe20102250 0x7efe2019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe20101890 0x7efe20197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 --2- 172.21.15.110:0/1452169041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe201006a0 0x7efe20197990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 >> 172.21.15.110:0/1452169041 conn(0x7efe200fbc30 msgr2=0x7efe200fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:10.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 shutdown_connections 2024-04-25T00:00:10.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.224+0000 7efe26676640 1 -- 172.21.15.110:0/1452169041 wait complete. 2024-04-25T00:00:10.236 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-04-25T00:00:08.507884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595456,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":132,"num_read_kb":114,"num_write":282,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":234,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351848,"kb_used_data":10968,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589976,"statfs":{"total":1151940427776,"available":1151580135424,"internally_reserved":0,"allocated":11231232,"data_stored":7018545,"data_compressed":108435,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5088,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":101,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001447"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.144251+0000","last_change":"2024-04-25T00:00:03.143150+0000","last_active":"2024-04-25T00:00:03.144251+0000","last_peered":"2024-04-25T00:00:03.144251+0000","last_clean":"2024-04-25T00:00:03.144251+0000","last_became_active":"2024-04-25T00:00:03.142344+0000","last_became_peered":"2024-04-25T00:00:03.142344+0000","last_unstale":"2024-04-25T00:00:03.144251+0000","last_undegraded":"2024-04-25T00:00:03.144251+0000","last_fullsized":"2024-04-25T00:00:03.144251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:41.257447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239163+0000","last_change":"2024-04-25T00:00:03.145454+0000","last_active":"2024-04-25T00:00:04.239163+0000","last_peered":"2024-04-25T00:00:04.239163+0000","last_clean":"2024-04-25T00:00:04.239163+0000","last_became_active":"2024-04-25T00:00:03.144905+0000","last_became_peered":"2024-04-25T00:00:03.144905+0000","last_unstale":"2024-04-25T00:00:04.239163+0000","last_undegraded":"2024-04-25T00:00:04.239163+0000","last_fullsized":"2024-04-25T00:00:04.239163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:39.583313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239000+0000","last_change":"2024-04-25T00:00:03.141770+0000","last_active":"2024-04-25T00:00:04.239000+0000","last_peered":"2024-04-25T00:00:04.239000+0000","last_clean":"2024-04-25T00:00:04.239000+0000","last_became_active":"2024-04-25T00:00:03.141468+0000","last_became_peered":"2024-04-25T00:00:03.141468+0000","last_unstale":"2024-04-25T00:00:04.239000+0000","last_undegraded":"2024-04-25T00:00:04.239000+0000","last_fullsized":"2024-04-25T00:00:04.239000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:33.112112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240201+0000","last_change":"2024-04-25T00:00:03.142992+0000","last_active":"2024-04-25T00:00:04.240201+0000","last_peered":"2024-04-25T00:00:04.240201+0000","last_clean":"2024-04-25T00:00:04.240201+0000","last_became_active":"2024-04-25T00:00:03.142692+0000","last_became_peered":"2024-04-25T00:00:03.142692+0000","last_unstale":"2024-04-25T00:00:04.240201+0000","last_undegraded":"2024-04-25T00:00:04.240201+0000","last_fullsized":"2024-04-25T00:00:04.240201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:21.513270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.148037+0000","last_change":"2024-04-25T00:00:03.146405+0000","last_active":"2024-04-25T00:00:03.148037+0000","last_peered":"2024-04-25T00:00:03.148037+0000","last_clean":"2024-04-25T00:00:03.148037+0000","last_became_active":"2024-04-25T00:00:03.145520+0000","last_became_peered":"2024-04-25T00:00:03.145520+0000","last_unstale":"2024-04-25T00:00:03.148037+0000","last_undegraded":"2024-04-25T00:00:03.148037+0000","last_fullsized":"2024-04-25T00:00:03.148037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:05.481290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128744+0000","last_change":"2024-04-25T00:00:03.142973+0000","last_active":"2024-04-25T00:00:04.128744+0000","last_peered":"2024-04-25T00:00:04.128744+0000","last_clean":"2024-04-25T00:00:04.128744+0000","last_became_active":"2024-04-25T00:00:03.142091+0000","last_became_peered":"2024-04-25T00:00:03.142091+0000","last_unstale":"2024-04-25T00:00:04.128744+0000","last_undegraded":"2024-04-25T00:00:04.128744+0000","last_fullsized":"2024-04-25T00:00:04.128744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:01.948299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239151+0000","last_change":"2024-04-25T00:00:03.148383+0000","last_active":"2024-04-25T00:00:04.239151+0000","last_peered":"2024-04-25T00:00:04.239151+0000","last_clean":"2024-04-25T00:00:04.239151+0000","last_became_active":"2024-04-25T00:00:03.148105+0000","last_became_peered":"2024-04-25T00:00:03.148105+0000","last_unstale":"2024-04-25T00:00:04.239151+0000","last_undegraded":"2024-04-25T00:00:04.239151+0000","last_fullsized":"2024-04-25T00:00:04.239151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:31.902600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238989+0000","last_change":"2024-04-25T00:00:03.151575+0000","last_active":"2024-04-25T00:00:04.238989+0000","last_peered":"2024-04-25T00:00:04.238989+0000","last_clean":"2024-04-25T00:00:04.238989+0000","last_became_active":"2024-04-25T00:00:03.151269+0000","last_became_peered":"2024-04-25T00:00:03.151269+0000","last_unstale":"2024-04-25T00:00:04.238989+0000","last_undegraded":"2024-04-25T00:00:04.238989+0000","last_fullsized":"2024-04-25T00:00:04.238989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:16.561180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239396+0000","last_change":"2024-04-25T00:00:03.140175+0000","last_active":"2024-04-25T00:00:04.239396+0000","last_peered":"2024-04-25T00:00:04.239396+0000","last_clean":"2024-04-25T00:00:04.239396+0000","last_became_active":"2024-04-25T00:00:03.139838+0000","last_became_peered":"2024-04-25T00:00:03.139838+0000","last_unstale":"2024-04-25T00:00:04.239396+0000","last_undegraded":"2024-04-25T00:00:04.239396+0000","last_fullsized":"2024-04-25T00:00:04.239396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:16.768025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.129090+0000","last_change":"2024-04-25T00:00:03.143149+0000","last_active":"2024-04-25T00:00:04.129090+0000","last_peered":"2024-04-25T00:00:04.129090+0000","last_clean":"2024-04-25T00:00:04.129090+0000","last_became_active":"2024-04-25T00:00:03.142799+0000","last_became_peered":"2024-04-25T00:00:03.142799+0000","last_unstale":"2024-04-25T00:00:04.129090+0000","last_undegraded":"2024-04-25T00:00:04.129090+0000","last_fullsized":"2024-04-25T00:00:04.129090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:38.941329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240368+0000","last_change":"2024-04-25T00:00:03.141519+0000","last_active":"2024-04-25T00:00:04.240368+0000","last_peered":"2024-04-25T00:00:04.240368+0000","last_clean":"2024-04-25T00:00:04.240368+0000","last_became_active":"2024-04-25T00:00:03.140452+0000","last_became_peered":"2024-04-25T00:00:03.140452+0000","last_unstale":"2024-04-25T00:00:04.240368+0000","last_undegraded":"2024-04-25T00:00:04.240368+0000","last_fullsized":"2024-04-25T00:00:04.240368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:17.552403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240071+0000","last_change":"2024-04-25T00:00:03.149727+0000","last_active":"2024-04-25T00:00:04.240071+0000","last_peered":"2024-04-25T00:00:04.240071+0000","last_clean":"2024-04-25T00:00:04.240071+0000","last_became_active":"2024-04-25T00:00:03.149205+0000","last_became_peered":"2024-04-25T00:00:03.149205+0000","last_unstale":"2024-04-25T00:00:04.240071+0000","last_undegraded":"2024-04-25T00:00:04.240071+0000","last_fullsized":"2024-04-25T00:00:04.240071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:55.782750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127848+0000","last_change":"2024-04-25T00:00:03.139219+0000","last_active":"2024-04-25T00:00:04.127848+0000","last_peered":"2024-04-25T00:00:04.127848+0000","last_clean":"2024-04-25T00:00:04.127848+0000","last_became_active":"2024-04-25T00:00:03.138883+0000","last_became_peered":"2024-04-25T00:00:03.138883+0000","last_unstale":"2024-04-25T00:00:04.127848+0000","last_undegraded":"2024-04-25T00:00:04.127848+0000","last_fullsized":"2024-04-25T00:00:04.127848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.744932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238771+0000","last_change":"2024-04-25T00:00:03.144473+0000","last_active":"2024-04-25T00:00:04.238771+0000","last_peered":"2024-04-25T00:00:04.238771+0000","last_clean":"2024-04-25T00:00:04.238771+0000","last_became_active":"2024-04-25T00:00:03.144166+0000","last_became_peered":"2024-04-25T00:00:03.144166+0000","last_unstale":"2024-04-25T00:00:04.238771+0000","last_undegraded":"2024-04-25T00:00:04.238771+0000","last_fullsized":"2024-04-25T00:00:04.238771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:25.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238482+0000","last_change":"2024-04-25T00:00:03.143292+0000","last_active":"2024-04-25T00:00:04.238482+0000","last_peered":"2024-04-25T00:00:04.238482+0000","last_clean":"2024-04-25T00:00:04.238482+0000","last_became_active":"2024-04-25T00:00:03.142993+0000","last_became_peered":"2024-04-25T00:00:03.142993+0000","last_unstale":"2024-04-25T00:00:04.238482+0000","last_undegraded":"2024-04-25T00:00:04.238482+0000","last_fullsized":"2024-04-25T00:00:04.238482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.841015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239242+0000","last_change":"2024-04-25T00:00:03.148117+0000","last_active":"2024-04-25T00:00:04.239242+0000","last_peered":"2024-04-25T00:00:04.239242+0000","last_clean":"2024-04-25T00:00:04.239242+0000","last_became_active":"2024-04-25T00:00:03.147670+0000","last_became_peered":"2024-04-25T00:00:03.147670+0000","last_unstale":"2024-04-25T00:00:04.239242+0000","last_undegraded":"2024-04-25T00:00:04.239242+0000","last_fullsized":"2024-04-25T00:00:04.239242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:01.716253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.144271+0000","last_change":"2024-04-25T00:00:03.143452+0000","last_active":"2024-04-25T00:00:03.144271+0000","last_peered":"2024-04-25T00:00:03.144271+0000","last_clean":"2024-04-25T00:00:03.144271+0000","last_became_active":"2024-04-25T00:00:03.142379+0000","last_became_peered":"2024-04-25T00:00:03.142379+0000","last_unstale":"2024-04-25T00:00:03.144271+0000","last_undegraded":"2024-04-25T00:00:03.144271+0000","last_fullsized":"2024-04-25T00:00:03.144271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:05.816682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239367+0000","last_change":"2024-04-25T00:00:03.138570+0000","last_active":"2024-04-25T00:00:04.239367+0000","last_peered":"2024-04-25T00:00:04.239367+0000","last_clean":"2024-04-25T00:00:04.239367+0000","last_became_active":"2024-04-25T00:00:03.138198+0000","last_became_peered":"2024-04-25T00:00:03.138198+0000","last_unstale":"2024-04-25T00:00:04.239367+0000","last_undegraded":"2024-04-25T00:00:04.239367+0000","last_fullsized":"2024-04-25T00:00:04.239367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:47.794135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.129133+0000","last_change":"2024-04-25T00:00:03.143152+0000","last_active":"2024-04-25T00:00:04.129133+0000","last_peered":"2024-04-25T00:00:04.129133+0000","last_clean":"2024-04-25T00:00:04.129133+0000","last_became_active":"2024-04-25T00:00:03.142812+0000","last_became_peered":"2024-04-25T00:00:03.142812+0000","last_unstale":"2024-04-25T00:00:04.129133+0000","last_undegraded":"2024-04-25T00:00:04.129133+0000","last_fullsized":"2024-04-25T00:00:04.129133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:58.808783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238607+0000","last_change":"2024-04-25T00:00:02.525740+0000","last_active":"2024-04-25T00:00:04.238607+0000","last_peered":"2024-04-25T00:00:04.238607+0000","last_clean":"2024-04-25T00:00:04.238607+0000","last_became_active":"2024-04-25T00:00:02.525429+0000","last_became_peered":"2024-04-25T00:00:02.525429+0000","last_unstale":"2024-04-25T00:00:04.238607+0000","last_undegraded":"2024-04-25T00:00:04.238607+0000","last_fullsized":"2024-04-25T00:00:04.238607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:43.585678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239381+0000","last_change":"2024-04-25T00:00:03.151963+0000","last_active":"2024-04-25T00:00:04.239381+0000","last_peered":"2024-04-25T00:00:04.239381+0000","last_clean":"2024-04-25T00:00:04.239381+0000","last_became_active":"2024-04-25T00:00:03.151059+0000","last_became_peered":"2024-04-25T00:00:03.151059+0000","last_unstale":"2024-04-25T00:00:04.239381+0000","last_undegraded":"2024-04-25T00:00:04.239381+0000","last_fullsized":"2024-04-25T00:00:04.239381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:37.834035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239548+0000","last_change":"2024-04-25T00:00:02.534181+0000","last_active":"2024-04-25T00:00:04.239548+0000","last_peered":"2024-04-25T00:00:04.239548+0000","last_clean":"2024-04-25T00:00:04.239548+0000","last_became_active":"2024-04-25T00:00:02.533848+0000","last_became_peered":"2024-04-25T00:00:02.533848+0000","last_unstale":"2024-04-25T00:00:04.239548+0000","last_undegraded":"2024-04-25T00:00:04.239548+0000","last_fullsized":"2024-04-25T00:00:04.239548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:25.276477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239356+0000","last_change":"2024-04-25T00:00:03.142038+0000","last_active":"2024-04-25T00:00:04.239356+0000","last_peered":"2024-04-25T00:00:04.239356+0000","last_clean":"2024-04-25T00:00:04.239356+0000","last_became_active":"2024-04-25T00:00:03.141705+0000","last_became_peered":"2024-04-25T00:00:03.141705+0000","last_unstale":"2024-04-25T00:00:04.239356+0000","last_undegraded":"2024-04-25T00:00:04.239356+0000","last_fullsized":"2024-04-25T00:00:04.239356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.382031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.531036+0000","last_change":"2024-04-25T00:00:02.526339+0000","last_active":"2024-04-25T00:00:03.531036+0000","last_peered":"2024-04-25T00:00:03.531036+0000","last_clean":"2024-04-25T00:00:03.531036+0000","last_became_active":"2024-04-25T00:00:02.525999+0000","last_became_peered":"2024-04-25T00:00:02.525999+0000","last_unstale":"2024-04-25T00:00:03.531036+0000","last_undegraded":"2024-04-25T00:00:03.531036+0000","last_fullsized":"2024-04-25T00:00:03.531036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:52.997821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238676+0000","last_change":"2024-04-25T00:00:03.143896+0000","last_active":"2024-04-25T00:00:04.238676+0000","last_peered":"2024-04-25T00:00:04.238676+0000","last_clean":"2024-04-25T00:00:04.238676+0000","last_became_active":"2024-04-25T00:00:03.141311+0000","last_became_peered":"2024-04-25T00:00:03.141311+0000","last_unstale":"2024-04-25T00:00:04.238676+0000","last_undegraded":"2024-04-25T00:00:04.238676+0000","last_fullsized":"2024-04-25T00:00:04.238676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:57.819319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.530961+0000","last_change":"2024-04-25T00:00:02.523497+0000","last_active":"2024-04-25T00:00:03.530961+0000","last_peered":"2024-04-25T00:00:03.530961+0000","last_clean":"2024-04-25T00:00:03.530961+0000","last_became_active":"2024-04-25T00:00:02.523153+0000","last_became_peered":"2024-04-25T00:00:02.523153+0000","last_unstale":"2024-04-25T00:00:03.530961+0000","last_undegraded":"2024-04-25T00:00:03.530961+0000","last_fullsized":"2024-04-25T00:00:03.530961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:44.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239507+0000","last_change":"2024-04-25T00:00:03.140867+0000","last_active":"2024-04-25T00:00:04.239507+0000","last_peered":"2024-04-25T00:00:04.239507+0000","last_clean":"2024-04-25T00:00:04.239507+0000","last_became_active":"2024-04-25T00:00:03.140545+0000","last_became_peered":"2024-04-25T00:00:03.140545+0000","last_unstale":"2024-04-25T00:00:04.239507+0000","last_undegraded":"2024-04-25T00:00:04.239507+0000","last_fullsized":"2024-04-25T00:00:04.239507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:46.928359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554801+0000","last_change":"2024-04-25T00:00:02.528897+0000","last_active":"2024-04-25T00:00:05.554801+0000","last_peered":"2024-04-25T00:00:05.554801+0000","last_clean":"2024-04-25T00:00:05.554801+0000","last_became_active":"2024-04-25T00:00:02.528584+0000","last_became_peered":"2024-04-25T00:00:02.528584+0000","last_unstale":"2024-04-25T00:00:05.554801+0000","last_undegraded":"2024-04-25T00:00:05.554801+0000","last_fullsized":"2024-04-25T00:00:05.554801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:23.437956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.150598+0000","last_change":"2024-04-25T00:00:03.147692+0000","last_active":"2024-04-25T00:00:03.150598+0000","last_peered":"2024-04-25T00:00:03.150598+0000","last_clean":"2024-04-25T00:00:03.150598+0000","last_became_active":"2024-04-25T00:00:03.147183+0000","last_became_peered":"2024-04-25T00:00:03.147183+0000","last_unstale":"2024-04-25T00:00:03.150598+0000","last_undegraded":"2024-04-25T00:00:03.150598+0000","last_fullsized":"2024-04-25T00:00:03.150598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:10:07.847896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.559274+0000","last_change":"2024-04-25T00:00:02.522387+0000","last_active":"2024-04-25T00:00:06.559274+0000","last_peered":"2024-04-25T00:00:06.559274+0000","last_clean":"2024-04-25T00:00:06.559274+0000","last_became_active":"2024-04-25T00:00:02.522093+0000","last_became_peered":"2024-04-25T00:00:02.522093+0000","last_unstale":"2024-04-25T00:00:06.559274+0000","last_undegraded":"2024-04-25T00:00:06.559274+0000","last_fullsized":"2024-04-25T00:00:06.559274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:33.393915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239511+0000","last_change":"2024-04-25T00:00:03.146277+0000","last_active":"2024-04-25T00:00:04.239511+0000","last_peered":"2024-04-25T00:00:04.239511+0000","last_clean":"2024-04-25T00:00:04.239511+0000","last_became_active":"2024-04-25T00:00:03.145846+0000","last_became_peered":"2024-04-25T00:00:03.145846+0000","last_unstale":"2024-04-25T00:00:04.239511+0000","last_undegraded":"2024-04-25T00:00:04.239511+0000","last_fullsized":"2024-04-25T00:00:04.239511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.692246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.558430+0000","last_change":"2024-04-25T00:00:02.528405+0000","last_active":"2024-04-25T00:00:06.558430+0000","last_peered":"2024-04-25T00:00:06.558430+0000","last_clean":"2024-04-25T00:00:06.558430+0000","last_became_active":"2024-04-25T00:00:02.528115+0000","last_became_peered":"2024-04-25T00:00:02.528115+0000","last_unstale":"2024-04-25T00:00:06.558430+0000","last_undegraded":"2024-04-25T00:00:06.558430+0000","last_fullsized":"2024-04-25T00:00:06.558430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:51.382361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238306+0000","last_change":"2024-04-25T00:00:03.144266+0000","last_active":"2024-04-25T00:00:04.238306+0000","last_peered":"2024-04-25T00:00:04.238306+0000","last_clean":"2024-04-25T00:00:04.238306+0000","last_became_active":"2024-04-25T00:00:03.141704+0000","last_became_peered":"2024-04-25T00:00:03.141704+0000","last_unstale":"2024-04-25T00:00:04.238306+0000","last_undegraded":"2024-04-25T00:00:04.238306+0000","last_fullsized":"2024-04-25T00:00:04.238306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:14.161783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238781+0000","last_change":"2024-04-25T00:00:02.528134+0000","last_active":"2024-04-25T00:00:04.238781+0000","last_peered":"2024-04-25T00:00:04.238781+0000","last_clean":"2024-04-25T00:00:04.238781+0000","last_became_active":"2024-04-25T00:00:02.527830+0000","last_became_peered":"2024-04-25T00:00:02.527830+0000","last_unstale":"2024-04-25T00:00:04.238781+0000","last_undegraded":"2024-04-25T00:00:04.238781+0000","last_fullsized":"2024-04-25T00:00:04.238781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:46.846244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239053+0000","last_change":"2024-04-25T00:00:03.143505+0000","last_active":"2024-04-25T00:00:04.239053+0000","last_peered":"2024-04-25T00:00:04.239053+0000","last_clean":"2024-04-25T00:00:04.239053+0000","last_became_active":"2024-04-25T00:00:03.143131+0000","last_became_peered":"2024-04-25T00:00:03.143131+0000","last_unstale":"2024-04-25T00:00:04.239053+0000","last_undegraded":"2024-04-25T00:00:04.239053+0000","last_fullsized":"2024-04-25T00:00:04.239053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:23.743162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554221+0000","last_change":"2024-04-25T00:00:02.525978+0000","last_active":"2024-04-25T00:00:05.554221+0000","last_peered":"2024-04-25T00:00:05.554221+0000","last_clean":"2024-04-25T00:00:05.554221+0000","last_became_active":"2024-04-25T00:00:02.525641+0000","last_became_peered":"2024-04-25T00:00:02.525641+0000","last_unstale":"2024-04-25T00:00:05.554221+0000","last_undegraded":"2024-04-25T00:00:05.554221+0000","last_fullsized":"2024-04-25T00:00:05.554221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:58.508573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238917+0000","last_change":"2024-04-25T00:00:03.143734+0000","last_active":"2024-04-25T00:00:04.238917+0000","last_peered":"2024-04-25T00:00:04.238917+0000","last_clean":"2024-04-25T00:00:04.238917+0000","last_became_active":"2024-04-25T00:00:03.143402+0000","last_became_peered":"2024-04-25T00:00:03.143402+0000","last_unstale":"2024-04-25T00:00:04.238917+0000","last_undegraded":"2024-04-25T00:00:04.238917+0000","last_fullsized":"2024-04-25T00:00:04.238917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:36.608803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.553889+0000","last_change":"2024-04-25T00:00:02.530567+0000","last_active":"2024-04-25T00:00:05.553889+0000","last_peered":"2024-04-25T00:00:05.553889+0000","last_clean":"2024-04-25T00:00:05.553889+0000","last_became_active":"2024-04-25T00:00:02.530066+0000","last_became_peered":"2024-04-25T00:00:02.530066+0000","last_unstale":"2024-04-25T00:00:05.553889+0000","last_undegraded":"2024-04-25T00:00:05.553889+0000","last_fullsized":"2024-04-25T00:00:05.553889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:19.892052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239299+0000","last_change":"2024-04-25T00:00:03.146905+0000","last_active":"2024-04-25T00:00:04.239299+0000","last_peered":"2024-04-25T00:00:04.239299+0000","last_clean":"2024-04-25T00:00:04.239299+0000","last_became_active":"2024-04-25T00:00:03.146080+0000","last_became_peered":"2024-04-25T00:00:03.146080+0000","last_unstale":"2024-04-25T00:00:04.239299+0000","last_undegraded":"2024-04-25T00:00:04.239299+0000","last_fullsized":"2024-04-25T00:00:04.239299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:42.764082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239677+0000","last_change":"2024-04-25T00:00:02.527667+0000","last_active":"2024-04-25T00:00:04.239677+0000","last_peered":"2024-04-25T00:00:04.239677+0000","last_clean":"2024-04-25T00:00:04.239677+0000","last_became_active":"2024-04-25T00:00:02.527333+0000","last_became_peered":"2024-04-25T00:00:02.527333+0000","last_unstale":"2024-04-25T00:00:04.239677+0000","last_undegraded":"2024-04-25T00:00:04.239677+0000","last_fullsized":"2024-04-25T00:00:04.239677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:15.982704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239060+0000","last_change":"2024-04-25T00:00:03.145856+0000","last_active":"2024-04-25T00:00:04.239060+0000","last_peered":"2024-04-25T00:00:04.239060+0000","last_clean":"2024-04-25T00:00:04.239060+0000","last_became_active":"2024-04-25T00:00:03.145406+0000","last_became_peered":"2024-04-25T00:00:03.145406+0000","last_unstale":"2024-04-25T00:00:04.239060+0000","last_undegraded":"2024-04-25T00:00:04.239060+0000","last_fullsized":"2024-04-25T00:00:04.239060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:28.318575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:07.561685+0000","last_change":"2024-04-25T00:00:02.522294+0000","last_active":"2024-04-25T00:00:07.561685+0000","last_peered":"2024-04-25T00:00:07.561685+0000","last_clean":"2024-04-25T00:00:07.561685+0000","last_became_active":"2024-04-25T00:00:02.521776+0000","last_became_peered":"2024-04-25T00:00:02.521776+0000","last_unstale":"2024-04-25T00:00:07.561685+0000","last_undegraded":"2024-04-25T00:00:07.561685+0000","last_fullsized":"2024-04-25T00:00:07.561685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:07.677118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.146533+0000","last_change":"2024-04-25T00:00:03.143877+0000","last_active":"2024-04-25T00:00:03.146533+0000","last_peered":"2024-04-25T00:00:03.146533+0000","last_clean":"2024-04-25T00:00:03.146533+0000","last_became_active":"2024-04-25T00:00:03.143558+0000","last_became_peered":"2024-04-25T00:00:03.143558+0000","last_unstale":"2024-04-25T00:00:03.146533+0000","last_undegraded":"2024-04-25T00:00:03.146533+0000","last_fullsized":"2024-04-25T00:00:03.146533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:43.373536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.557497+0000","last_change":"2024-04-25T00:00:02.520760+0000","last_active":"2024-04-25T00:00:06.557497+0000","last_peered":"2024-04-25T00:00:06.557497+0000","last_clean":"2024-04-25T00:00:06.557497+0000","last_became_active":"2024-04-25T00:00:02.520417+0000","last_became_peered":"2024-04-25T00:00:02.520417+0000","last_unstale":"2024-04-25T00:00:06.557497+0000","last_undegraded":"2024-04-25T00:00:06.557497+0000","last_fullsized":"2024-04-25T00:00:06.557497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:53.895653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127633+0000","last_change":"2024-04-25T00:00:03.144157+0000","last_active":"2024-04-25T00:00:04.127633+0000","last_peered":"2024-04-25T00:00:04.127633+0000","last_clean":"2024-04-25T00:00:04.127633+0000","last_became_active":"2024-04-25T00:00:03.143816+0000","last_became_peered":"2024-04-25T00:00:03.143816+0000","last_unstale":"2024-04-25T00:00:04.127633+0000","last_undegraded":"2024-04-25T00:00:04.127633+0000","last_fullsized":"2024-04-25T00:00:04.127633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:45.007842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554504+0000","last_change":"2024-04-25T00:00:02.534393+0000","last_active":"2024-04-25T00:00:05.554504+0000","last_peered":"2024-04-25T00:00:05.554504+0000","last_clean":"2024-04-25T00:00:05.554504+0000","last_became_active":"2024-04-25T00:00:02.534045+0000","last_became_peered":"2024-04-25T00:00:02.534045+0000","last_unstale":"2024-04-25T00:00:05.554504+0000","last_undegraded":"2024-04-25T00:00:05.554504+0000","last_fullsized":"2024-04-25T00:00:05.554504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:35.547065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240074+0000","last_change":"2024-04-25T00:00:03.149787+0000","last_active":"2024-04-25T00:00:04.240074+0000","last_peered":"2024-04-25T00:00:04.240074+0000","last_clean":"2024-04-25T00:00:04.240074+0000","last_became_active":"2024-04-25T00:00:03.149482+0000","last_became_peered":"2024-04-25T00:00:03.149482+0000","last_unstale":"2024-04-25T00:00:04.240074+0000","last_undegraded":"2024-04-25T00:00:04.240074+0000","last_fullsized":"2024-04-25T00:00:04.240074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:32.278190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.558252+0000","last_change":"2024-04-25T00:00:02.520357+0000","last_active":"2024-04-25T00:00:06.558252+0000","last_peered":"2024-04-25T00:00:06.558252+0000","last_clean":"2024-04-25T00:00:06.558252+0000","last_became_active":"2024-04-25T00:00:02.519926+0000","last_became_peered":"2024-04-25T00:00:02.519926+0000","last_unstale":"2024-04-25T00:00:06.558252+0000","last_undegraded":"2024-04-25T00:00:06.558252+0000","last_fullsized":"2024-04-25T00:00:06.558252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:52.438584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240147+0000","last_change":"2024-04-25T00:00:03.147560+0000","last_active":"2024-04-25T00:00:04.240147+0000","last_peered":"2024-04-25T00:00:04.240147+0000","last_clean":"2024-04-25T00:00:04.240147+0000","last_became_active":"2024-04-25T00:00:03.147234+0000","last_became_peered":"2024-04-25T00:00:03.147234+0000","last_unstale":"2024-04-25T00:00:04.240147+0000","last_undegraded":"2024-04-25T00:00:04.240147+0000","last_fullsized":"2024-04-25T00:00:04.240147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:33.737763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.531641+0000","last_change":"2024-04-25T00:00:02.528614+0000","last_active":"2024-04-25T00:00:03.531641+0000","last_peered":"2024-04-25T00:00:03.531641+0000","last_clean":"2024-04-25T00:00:03.531641+0000","last_became_active":"2024-04-25T00:00:02.528203+0000","last_became_peered":"2024-04-25T00:00:02.528203+0000","last_unstale":"2024-04-25T00:00:03.531641+0000","last_undegraded":"2024-04-25T00:00:03.531641+0000","last_fullsized":"2024-04-25T00:00:03.531641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:33.116699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239244+0000","last_change":"2024-04-25T00:00:03.145009+0000","last_active":"2024-04-25T00:00:04.239244+0000","last_peered":"2024-04-25T00:00:04.239244+0000","last_clean":"2024-04-25T00:00:04.239244+0000","last_became_active":"2024-04-25T00:00:03.144575+0000","last_became_peered":"2024-04-25T00:00:03.144575+0000","last_unstale":"2024-04-25T00:00:04.239244+0000","last_undegraded":"2024-04-25T00:00:04.239244+0000","last_fullsized":"2024-04-25T00:00:04.239244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.219880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.530133+0000","last_change":"2024-04-25T00:00:02.524951+0000","last_active":"2024-04-25T00:00:03.530133+0000","last_peered":"2024-04-25T00:00:03.530133+0000","last_clean":"2024-04-25T00:00:03.530133+0000","last_became_active":"2024-04-25T00:00:02.524624+0000","last_became_peered":"2024-04-25T00:00:02.524624+0000","last_unstale":"2024-04-25T00:00:03.530133+0000","last_undegraded":"2024-04-25T00:00:03.530133+0000","last_fullsized":"2024-04-25T00:00:03.530133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:23.381521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.148198+0000","last_change":"2024-04-25T00:00:03.145446+0000","last_active":"2024-04-25T00:00:03.148198+0000","last_peered":"2024-04-25T00:00:03.148198+0000","last_clean":"2024-04-25T00:00:03.148198+0000","last_became_active":"2024-04-25T00:00:03.145125+0000","last_became_peered":"2024-04-25T00:00:03.145125+0000","last_unstale":"2024-04-25T00:00:03.148198+0000","last_undegraded":"2024-04-25T00:00:03.148198+0000","last_fullsized":"2024-04-25T00:00:03.148198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:49.845075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238725+0000","last_change":"2024-04-25T00:00:02.527499+0000","last_active":"2024-04-25T00:00:04.238725+0000","last_peered":"2024-04-25T00:00:04.238725+0000","last_clean":"2024-04-25T00:00:04.238725+0000","last_became_active":"2024-04-25T00:00:02.527151+0000","last_became_peered":"2024-04-25T00:00:02.527151+0000","last_unstale":"2024-04-25T00:00:04.238725+0000","last_undegraded":"2024-04-25T00:00:04.238725+0000","last_fullsized":"2024-04-25T00:00:04.238725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:09.013930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.129109+0000","last_change":"2024-04-25T00:00:03.148231+0000","last_active":"2024-04-25T00:00:04.129109+0000","last_peered":"2024-04-25T00:00:04.129109+0000","last_clean":"2024-04-25T00:00:04.129109+0000","last_became_active":"2024-04-25T00:00:03.147891+0000","last_became_peered":"2024-04-25T00:00:03.147891+0000","last_unstale":"2024-04-25T00:00:04.129109+0000","last_undegraded":"2024-04-25T00:00:04.129109+0000","last_fullsized":"2024-04-25T00:00:04.129109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:42.784699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240238+0000","last_change":"2024-04-25T00:00:02.528234+0000","last_active":"2024-04-25T00:00:04.240238+0000","last_peered":"2024-04-25T00:00:04.240238+0000","last_clean":"2024-04-25T00:00:04.240238+0000","last_became_active":"2024-04-25T00:00:02.527875+0000","last_became_peered":"2024-04-25T00:00:02.527875+0000","last_unstale":"2024-04-25T00:00:04.240238+0000","last_undegraded":"2024-04-25T00:00:04.240238+0000","last_fullsized":"2024-04-25T00:00:04.240238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:06.634497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128888+0000","last_change":"2024-04-25T00:00:03.144246+0000","last_active":"2024-04-25T00:00:04.128888+0000","last_peered":"2024-04-25T00:00:04.128888+0000","last_clean":"2024-04-25T00:00:04.128888+0000","last_became_active":"2024-04-25T00:00:03.143925+0000","last_became_peered":"2024-04-25T00:00:03.143925+0000","last_unstale":"2024-04-25T00:00:04.128888+0000","last_undegraded":"2024-04-25T00:00:04.128888+0000","last_fullsized":"2024-04-25T00:00:04.128888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:51.741275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239124+0000","last_change":"2024-04-25T00:00:03.141261+0000","last_active":"2024-04-25T00:00:04.239124+0000","last_peered":"2024-04-25T00:00:04.239124+0000","last_clean":"2024-04-25T00:00:04.239124+0000","last_became_active":"2024-04-25T00:00:03.137943+0000","last_became_peered":"2024-04-25T00:00:03.137943+0000","last_unstale":"2024-04-25T00:00:04.239124+0000","last_undegraded":"2024-04-25T00:00:04.239124+0000","last_fullsized":"2024-04-25T00:00:04.239124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:41.655749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.559557+0000","last_change":"2024-04-25T00:00:02.522048+0000","last_active":"2024-04-25T00:00:06.559557+0000","last_peered":"2024-04-25T00:00:06.559557+0000","last_clean":"2024-04-25T00:00:06.559557+0000","last_became_active":"2024-04-25T00:00:02.521755+0000","last_became_peered":"2024-04-25T00:00:02.521755+0000","last_unstale":"2024-04-25T00:00:06.559557+0000","last_undegraded":"2024-04-25T00:00:06.559557+0000","last_fullsized":"2024-04-25T00:00:06.559557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:58.602357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239320+0000","last_change":"2024-04-25T00:00:03.151428+0000","last_active":"2024-04-25T00:00:04.239320+0000","last_peered":"2024-04-25T00:00:04.239320+0000","last_clean":"2024-04-25T00:00:04.239320+0000","last_became_active":"2024-04-25T00:00:03.151005+0000","last_became_peered":"2024-04-25T00:00:03.151005+0000","last_unstale":"2024-04-25T00:00:04.239320+0000","last_undegraded":"2024-04-25T00:00:04.239320+0000","last_fullsized":"2024-04-25T00:00:04.239320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:28.044803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554997+0000","last_change":"2024-04-25T00:00:02.515714+0000","last_active":"2024-04-25T00:00:05.554997+0000","last_peered":"2024-04-25T00:00:05.554997+0000","last_clean":"2024-04-25T00:00:05.554997+0000","last_became_active":"2024-04-25T00:00:02.515373+0000","last_became_peered":"2024-04-25T00:00:02.515373+0000","last_unstale":"2024-04-25T00:00:05.554997+0000","last_undegraded":"2024-04-25T00:00:05.554997+0000","last_fullsized":"2024-04-25T00:00:05.554997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:34.344261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239409+0000","last_change":"2024-04-24T23:58:50.595622+0000","last_active":"2024-04-25T00:00:04.239409+0000","last_peered":"2024-04-25T00:00:04.239409+0000","last_clean":"2024-04-25T00:00:04.239409+0000","last_became_active":"2024-04-24T23:58:50.594971+0000","last_became_peered":"2024-04-24T23:58:50.594971+0000","last_unstale":"2024-04-25T00:00:04.239409+0000","last_undegraded":"2024-04-25T00:00:04.239409+0000","last_fullsized":"2024-04-25T00:00:04.239409+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:55:22.756734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:55:22.756734+0000","last_clean_scrub_stamp":"2024-04-24T23:55:22.756734+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:20.241007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127517+0000","last_change":"2024-04-25T00:00:03.135594+0000","last_active":"2024-04-25T00:00:04.127517+0000","last_peered":"2024-04-25T00:00:04.127517+0000","last_clean":"2024-04-25T00:00:04.127517+0000","last_became_active":"2024-04-25T00:00:03.135279+0000","last_became_peered":"2024-04-25T00:00:03.135279+0000","last_unstale":"2024-04-25T00:00:04.127517+0000","last_undegraded":"2024-04-25T00:00:04.127517+0000","last_fullsized":"2024-04-25T00:00:04.127517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:37.690119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.558627+0000","last_change":"2024-04-25T00:00:02.527294+0000","last_active":"2024-04-25T00:00:06.558627+0000","last_peered":"2024-04-25T00:00:06.558627+0000","last_clean":"2024-04-25T00:00:06.558627+0000","last_became_active":"2024-04-25T00:00:02.526979+0000","last_became_peered":"2024-04-25T00:00:02.526979+0000","last_unstale":"2024-04-25T00:00:06.558627+0000","last_undegraded":"2024-04-25T00:00:06.558627+0000","last_fullsized":"2024-04-25T00:00:06.558627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:04.784065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.134858+0000","last_change":"2024-04-25T00:00:02.527545+0000","last_active":"2024-04-25T00:00:03.134858+0000","last_peered":"2024-04-25T00:00:03.134858+0000","last_clean":"2024-04-25T00:00:03.134858+0000","last_became_active":"2024-04-25T00:00:02.527250+0000","last_became_peered":"2024-04-25T00:00:02.527250+0000","last_unstale":"2024-04-25T00:00:03.134858+0000","last_undegraded":"2024-04-25T00:00:03.134858+0000","last_fullsized":"2024-04-25T00:00:03.134858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:57.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127727+0000","last_change":"2024-04-25T00:00:03.139578+0000","last_active":"2024-04-25T00:00:04.127727+0000","last_peered":"2024-04-25T00:00:04.127727+0000","last_clean":"2024-04-25T00:00:04.127727+0000","last_became_active":"2024-04-25T00:00:03.139128+0000","last_became_peered":"2024-04-25T00:00:03.139128+0000","last_unstale":"2024-04-25T00:00:04.127727+0000","last_undegraded":"2024-04-25T00:00:04.127727+0000","last_fullsized":"2024-04-25T00:00:04.127727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:33.509293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.525292+0000","last_change":"2024-04-25T00:00:02.520616+0000","last_active":"2024-04-25T00:00:05.525292+0000","last_peered":"2024-04-25T00:00:05.525292+0000","last_clean":"2024-04-25T00:00:05.525292+0000","last_became_active":"2024-04-25T00:00:02.519057+0000","last_became_peered":"2024-04-25T00:00:02.519057+0000","last_unstale":"2024-04-25T00:00:05.525292+0000","last_undegraded":"2024-04-25T00:00:05.525292+0000","last_fullsized":"2024-04-25T00:00:05.525292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:25.847055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239078+0000","last_change":"2024-04-25T00:00:03.150010+0000","last_active":"2024-04-25T00:00:04.239078+0000","last_peered":"2024-04-25T00:00:04.239078+0000","last_clean":"2024-04-25T00:00:04.239078+0000","last_became_active":"2024-04-25T00:00:03.149677+0000","last_became_peered":"2024-04-25T00:00:03.149677+0000","last_unstale":"2024-04-25T00:00:04.239078+0000","last_undegraded":"2024-04-25T00:00:04.239078+0000","last_fullsized":"2024-04-25T00:00:04.239078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:18.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.551533+0000","last_change":"2024-04-25T00:00:02.533171+0000","last_active":"2024-04-25T00:00:05.551533+0000","last_peered":"2024-04-25T00:00:05.551533+0000","last_clean":"2024-04-25T00:00:05.551533+0000","last_became_active":"2024-04-25T00:00:02.532887+0000","last_became_peered":"2024-04-25T00:00:02.532887+0000","last_unstale":"2024-04-25T00:00:05.551533+0000","last_undegraded":"2024-04-25T00:00:05.551533+0000","last_fullsized":"2024-04-25T00:00:05.551533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:24.729809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239095+0000","last_change":"2024-04-25T00:00:03.147457+0000","last_active":"2024-04-25T00:00:04.239095+0000","last_peered":"2024-04-25T00:00:04.239095+0000","last_clean":"2024-04-25T00:00:04.239095+0000","last_became_active":"2024-04-25T00:00:03.147163+0000","last_became_peered":"2024-04-25T00:00:03.147163+0000","last_unstale":"2024-04-25T00:00:04.239095+0000","last_undegraded":"2024-04-25T00:00:04.239095+0000","last_fullsized":"2024-04-25T00:00:04.239095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:32.422006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554061+0000","last_change":"2024-04-25T00:00:02.527573+0000","last_active":"2024-04-25T00:00:05.554061+0000","last_peered":"2024-04-25T00:00:05.554061+0000","last_clean":"2024-04-25T00:00:05.554061+0000","last_became_active":"2024-04-25T00:00:02.527267+0000","last_became_peered":"2024-04-25T00:00:02.527267+0000","last_unstale":"2024-04-25T00:00:05.554061+0000","last_undegraded":"2024-04-25T00:00:05.554061+0000","last_fullsized":"2024-04-25T00:00:05.554061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:31.581529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.148030+0000","last_change":"2024-04-25T00:00:03.146602+0000","last_active":"2024-04-25T00:00:03.148030+0000","last_peered":"2024-04-25T00:00:03.148030+0000","last_clean":"2024-04-25T00:00:03.148030+0000","last_became_active":"2024-04-25T00:00:03.145480+0000","last_became_peered":"2024-04-25T00:00:03.145480+0000","last_unstale":"2024-04-25T00:00:03.148030+0000","last_undegraded":"2024-04-25T00:00:03.148030+0000","last_fullsized":"2024-04-25T00:00:03.148030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.319078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554920+0000","last_change":"2024-04-25T00:00:02.526567+0000","last_active":"2024-04-25T00:00:05.554920+0000","last_peered":"2024-04-25T00:00:05.554920+0000","last_clean":"2024-04-25T00:00:05.554920+0000","last_became_active":"2024-04-25T00:00:02.526234+0000","last_became_peered":"2024-04-25T00:00:02.526234+0000","last_unstale":"2024-04-25T00:00:05.554920+0000","last_undegraded":"2024-04-25T00:00:05.554920+0000","last_fullsized":"2024-04-25T00:00:05.554920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:14.964206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.141897+0000","last_change":"2024-04-25T00:00:03.140959+0000","last_active":"2024-04-25T00:00:03.141897+0000","last_peered":"2024-04-25T00:00:03.141897+0000","last_clean":"2024-04-25T00:00:03.141897+0000","last_became_active":"2024-04-25T00:00:03.140533+0000","last_became_peered":"2024-04-25T00:00:03.140533+0000","last_unstale":"2024-04-25T00:00:03.141897+0000","last_undegraded":"2024-04-25T00:00:03.141897+0000","last_fullsized":"2024-04-25T00:00:03.141897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:14.923767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128803+0000","last_change":"2024-04-25T00:00:02.527362+0000","last_active":"2024-04-25T00:00:04.128803+0000","last_peered":"2024-04-25T00:00:04.128803+0000","last_clean":"2024-04-25T00:00:04.128803+0000","last_became_active":"2024-04-25T00:00:02.527037+0000","last_became_peered":"2024-04-25T00:00:02.527037+0000","last_unstale":"2024-04-25T00:00:04.128803+0000","last_undegraded":"2024-04-25T00:00:04.128803+0000","last_fullsized":"2024-04-25T00:00:04.128803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:53.365117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239070+0000","last_change":"2024-04-25T00:00:03.149740+0000","last_active":"2024-04-25T00:00:04.239070+0000","last_peered":"2024-04-25T00:00:04.239070+0000","last_clean":"2024-04-25T00:00:04.239070+0000","last_became_active":"2024-04-25T00:00:03.149290+0000","last_became_peered":"2024-04-25T00:00:03.149290+0000","last_unstale":"2024-04-25T00:00:04.239070+0000","last_undegraded":"2024-04-25T00:00:04.239070+0000","last_fullsized":"2024-04-25T00:00:04.239070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.716974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.530478+0000","last_change":"2024-04-25T00:00:02.526823+0000","last_active":"2024-04-25T00:00:03.530478+0000","last_peered":"2024-04-25T00:00:03.530478+0000","last_clean":"2024-04-25T00:00:03.530478+0000","last_became_active":"2024-04-25T00:00:02.526529+0000","last_became_peered":"2024-04-25T00:00:02.526529+0000","last_unstale":"2024-04-25T00:00:03.530478+0000","last_undegraded":"2024-04-25T00:00:03.530478+0000","last_fullsized":"2024-04-25T00:00:03.530478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:13.907985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240466+0000","last_change":"2024-04-25T00:00:03.141973+0000","last_active":"2024-04-25T00:00:04.240466+0000","last_peered":"2024-04-25T00:00:04.240466+0000","last_clean":"2024-04-25T00:00:04.240466+0000","last_became_active":"2024-04-25T00:00:03.141560+0000","last_became_peered":"2024-04-25T00:00:03.141560+0000","last_unstale":"2024-04-25T00:00:04.240466+0000","last_undegraded":"2024-04-25T00:00:04.240466+0000","last_fullsized":"2024-04-25T00:00:04.240466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:03.002939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.560548+0000","last_change":"2024-04-25T00:00:02.533013+0000","last_active":"2024-04-25T00:00:06.560548+0000","last_peered":"2024-04-25T00:00:06.560548+0000","last_clean":"2024-04-25T00:00:06.560548+0000","last_became_active":"2024-04-25T00:00:02.532667+0000","last_became_peered":"2024-04-25T00:00:02.532667+0000","last_unstale":"2024-04-25T00:00:06.560548+0000","last_undegraded":"2024-04-25T00:00:06.560548+0000","last_fullsized":"2024-04-25T00:00:06.560548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:07.469136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240415+0000","last_change":"2024-04-25T00:00:03.140120+0000","last_active":"2024-04-25T00:00:04.240415+0000","last_peered":"2024-04-25T00:00:04.240415+0000","last_clean":"2024-04-25T00:00:04.240415+0000","last_became_active":"2024-04-25T00:00:03.139778+0000","last_became_peered":"2024-04-25T00:00:03.139778+0000","last_unstale":"2024-04-25T00:00:04.240415+0000","last_undegraded":"2024-04-25T00:00:04.240415+0000","last_fullsized":"2024-04-25T00:00:04.240415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:02.678303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239086+0000","last_change":"2024-04-25T00:00:02.534359+0000","last_active":"2024-04-25T00:00:04.239086+0000","last_peered":"2024-04-25T00:00:04.239086+0000","last_clean":"2024-04-25T00:00:04.239086+0000","last_became_active":"2024-04-25T00:00:02.534057+0000","last_became_peered":"2024-04-25T00:00:02.534057+0000","last_unstale":"2024-04-25T00:00:04.239086+0000","last_undegraded":"2024-04-25T00:00:04.239086+0000","last_fullsized":"2024-04-25T00:00:04.239086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:38.716263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239636+0000","last_change":"2024-04-25T00:00:03.146608+0000","last_active":"2024-04-25T00:00:04.239636+0000","last_peered":"2024-04-25T00:00:04.239636+0000","last_clean":"2024-04-25T00:00:04.239636+0000","last_became_active":"2024-04-25T00:00:03.146256+0000","last_became_peered":"2024-04-25T00:00:03.146256+0000","last_unstale":"2024-04-25T00:00:04.239636+0000","last_undegraded":"2024-04-25T00:00:04.239636+0000","last_fullsized":"2024-04-25T00:00:04.239636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:04.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127851+0000","last_change":"2024-04-25T00:00:02.524736+0000","last_active":"2024-04-25T00:00:04.127851+0000","last_peered":"2024-04-25T00:00:04.127851+0000","last_clean":"2024-04-25T00:00:04.127851+0000","last_became_active":"2024-04-25T00:00:02.523822+0000","last_became_peered":"2024-04-25T00:00:02.523822+0000","last_unstale":"2024-04-25T00:00:04.127851+0000","last_undegraded":"2024-04-25T00:00:04.127851+0000","last_fullsized":"2024-04-25T00:00:04.127851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:13.448915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240497+0000","last_change":"2024-04-25T00:00:03.141296+0000","last_active":"2024-04-25T00:00:04.240497+0000","last_peered":"2024-04-25T00:00:04.240497+0000","last_clean":"2024-04-25T00:00:04.240497+0000","last_became_active":"2024-04-25T00:00:03.139916+0000","last_became_peered":"2024-04-25T00:00:03.139916+0000","last_unstale":"2024-04-25T00:00:04.240497+0000","last_undegraded":"2024-04-25T00:00:04.240497+0000","last_fullsized":"2024-04-25T00:00:04.240497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:24.534210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.150469+0000","last_change":"2024-04-25T00:00:03.149427+0000","last_active":"2024-04-25T00:00:03.150469+0000","last_peered":"2024-04-25T00:00:03.150469+0000","last_clean":"2024-04-25T00:00:03.150469+0000","last_became_active":"2024-04-25T00:00:03.148445+0000","last_became_peered":"2024-04-25T00:00:03.148445+0000","last_unstale":"2024-04-25T00:00:03.150469+0000","last_undegraded":"2024-04-25T00:00:03.150469+0000","last_fullsized":"2024-04-25T00:00:03.150469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:13.387947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128752+0000","last_change":"2024-04-25T00:00:03.146994+0000","last_active":"2024-04-25T00:00:04.128752+0000","last_peered":"2024-04-25T00:00:04.128752+0000","last_clean":"2024-04-25T00:00:04.128752+0000","last_became_active":"2024-04-25T00:00:03.146677+0000","last_became_peered":"2024-04-25T00:00:03.146677+0000","last_unstale":"2024-04-25T00:00:04.128752+0000","last_undegraded":"2024-04-25T00:00:04.128752+0000","last_fullsized":"2024-04-25T00:00:04.128752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:33.883583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128798+0000","last_change":"2024-04-25T00:00:03.149428+0000","last_active":"2024-04-25T00:00:04.128798+0000","last_peered":"2024-04-25T00:00:04.128798+0000","last_clean":"2024-04-25T00:00:04.128798+0000","last_became_active":"2024-04-25T00:00:03.147978+0000","last_became_peered":"2024-04-25T00:00:03.147978+0000","last_unstale":"2024-04-25T00:00:04.128798+0000","last_undegraded":"2024-04-25T00:00:04.128798+0000","last_fullsized":"2024-04-25T00:00:04.128798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:34.684463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238925+0000","last_change":"2024-04-25T00:00:03.147528+0000","last_active":"2024-04-25T00:00:04.238925+0000","last_peered":"2024-04-25T00:00:04.238925+0000","last_clean":"2024-04-25T00:00:04.238925+0000","last_became_active":"2024-04-25T00:00:03.147182+0000","last_became_peered":"2024-04-25T00:00:03.147182+0000","last_unstale":"2024-04-25T00:00:04.238925+0000","last_undegraded":"2024-04-25T00:00:04.238925+0000","last_fullsized":"2024-04-25T00:00:04.238925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:27.592335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127946+0000","last_change":"2024-04-25T00:00:03.143274+0000","last_active":"2024-04-25T00:00:04.127946+0000","last_peered":"2024-04-25T00:00:04.127946+0000","last_clean":"2024-04-25T00:00:04.127946+0000","last_became_active":"2024-04-25T00:00:03.142384+0000","last_became_peered":"2024-04-25T00:00:03.142384+0000","last_unstale":"2024-04-25T00:00:04.127946+0000","last_undegraded":"2024-04-25T00:00:04.127946+0000","last_fullsized":"2024-04-25T00:00:04.127946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:41.118123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.150414+0000","last_change":"2024-04-25T00:00:03.147769+0000","last_active":"2024-04-25T00:00:03.150414+0000","last_peered":"2024-04-25T00:00:03.150414+0000","last_clean":"2024-04-25T00:00:03.150414+0000","last_became_active":"2024-04-25T00:00:03.147009+0000","last_became_peered":"2024-04-25T00:00:03.147009+0000","last_unstale":"2024-04-25T00:00:03.150414+0000","last_undegraded":"2024-04-25T00:00:03.150414+0000","last_fullsized":"2024-04-25T00:00:03.150414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:56.373847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.129268+0000","last_change":"2024-04-25T00:00:03.141519+0000","last_active":"2024-04-25T00:00:04.129268+0000","last_peered":"2024-04-25T00:00:04.129268+0000","last_clean":"2024-04-25T00:00:04.129268+0000","last_became_active":"2024-04-25T00:00:03.139391+0000","last_became_peered":"2024-04-25T00:00:03.139391+0000","last_unstale":"2024-04-25T00:00:04.129268+0000","last_undegraded":"2024-04-25T00:00:04.129268+0000","last_fullsized":"2024-04-25T00:00:04.129268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:13.043899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238914+0000","last_change":"2024-04-25T00:00:03.151311+0000","last_active":"2024-04-25T00:00:04.238914+0000","last_peered":"2024-04-25T00:00:04.238914+0000","last_clean":"2024-04-25T00:00:04.238914+0000","last_became_active":"2024-04-25T00:00:03.150992+0000","last_became_peered":"2024-04-25T00:00:03.150992+0000","last_unstale":"2024-04-25T00:00:04.238914+0000","last_undegraded":"2024-04-25T00:00:04.238914+0000","last_fullsized":"2024-04-25T00:00:04.238914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:47.572641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239200+0000","last_change":"2024-04-25T00:00:03.145623+0000","last_active":"2024-04-25T00:00:04.239200+0000","last_peered":"2024-04-25T00:00:04.239200+0000","last_clean":"2024-04-25T00:00:04.239200+0000","last_became_active":"2024-04-25T00:00:03.145260+0000","last_became_peered":"2024-04-25T00:00:03.145260+0000","last_unstale":"2024-04-25T00:00:04.239200+0000","last_undegraded":"2024-04-25T00:00:04.239200+0000","last_fullsized":"2024-04-25T00:00:04.239200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:16.985732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238477+0000","last_change":"2024-04-25T00:00:03.144957+0000","last_active":"2024-04-25T00:00:04.238477+0000","last_peered":"2024-04-25T00:00:04.238477+0000","last_clean":"2024-04-25T00:00:04.238477+0000","last_became_active":"2024-04-25T00:00:03.144631+0000","last_became_peered":"2024-04-25T00:00:03.144631+0000","last_unstale":"2024-04-25T00:00:04.238477+0000","last_undegraded":"2024-04-25T00:00:04.238477+0000","last_fullsized":"2024-04-25T00:00:04.238477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.636553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239170+0000","last_change":"2024-04-25T00:00:03.146574+0000","last_active":"2024-04-25T00:00:04.239170+0000","last_peered":"2024-04-25T00:00:04.239170+0000","last_clean":"2024-04-25T00:00:04.239170+0000","last_became_active":"2024-04-25T00:00:03.145095+0000","last_became_peered":"2024-04-25T00:00:03.145095+0000","last_unstale":"2024-04-25T00:00:04.239170+0000","last_undegraded":"2024-04-25T00:00:04.239170+0000","last_fullsized":"2024-04-25T00:00:04.239170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:38.060170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127893+0000","last_change":"2024-04-25T00:00:03.143958+0000","last_active":"2024-04-25T00:00:04.127893+0000","last_peered":"2024-04-25T00:00:04.127893+0000","last_clean":"2024-04-25T00:00:04.127893+0000","last_became_active":"2024-04-25T00:00:03.143621+0000","last_became_peered":"2024-04-25T00:00:03.143621+0000","last_unstale":"2024-04-25T00:00:04.127893+0000","last_undegraded":"2024-04-25T00:00:04.127893+0000","last_fullsized":"2024-04-25T00:00:04.127893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:21.833252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T00:00:03.144430+0000","last_change":"2024-04-25T00:00:03.143703+0000","last_active":"2024-04-25T00:00:03.144430+0000","last_peered":"2024-04-25T00:00:03.144430+0000","last_clean":"2024-04-25T00:00:03.144430+0000","last_became_active":"2024-04-25T00:00:03.143400+0000","last_became_peered":"2024-04-25T00:00:03.143400+0000","last_unstale":"2024-04-25T00:00:03.144430+0000","last_undegraded":"2024-04-25T00:00:03.144430+0000","last_fullsized":"2024-04-25T00:00:03.144430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:46.258101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284249,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5088,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":101,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":13528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383692,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":1030681,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":440789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:00:10.240 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:00:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.373+0000 7f947fd63640 1 -- 172.21.15.110:0/4286367989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94780ffa50 msgr2=0x7f94780ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.373+0000 7f947fd63640 1 --2- 172.21.15.110:0/4286367989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94780ffa50 0x7f94780ffe70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9468009930 tx=0x7f946802a420 comp rx=0 tx=0).stop 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.373+0000 7f947fd63640 1 -- 172.21.15.110:0/4286367989 shutdown_connections 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.373+0000 7f947fd63640 1 --2- 172.21.15.110:0/4286367989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94781033e0 0x7f9478109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.373+0000 7f947fd63640 1 --2- 172.21.15.110:0/4286367989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9478102a20 0x7f9478102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.373+0000 7f947fd63640 1 --2- 172.21.15.110:0/4286367989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94780ffa50 0x7f94780ffe70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.373+0000 7f947fd63640 1 -- 172.21.15.110:0/4286367989 >> 172.21.15.110:0/4286367989 conn(0x7f94780fb380 msgr2=0x7f94780fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 -- 172.21.15.110:0/4286367989 shutdown_connections 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 -- 172.21.15.110:0/4286367989 wait complete. 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 Processor -- start 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 -- start start 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94780ffa50 0x7f9478197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:10.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9478102a20 0x7f9478197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94781033e0 0x7f947819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f947810d2d0 con 0x7f9478102a20 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f947810d150 con 0x7f94780ffa50 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947fd63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f947810d450 con 0x7f94781033e0 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947d2fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9478102a20 0x7f9478197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947d2fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9478102a20 0x7f9478197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34600/0 (socket says 172.21.15.110:34600) 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947d2fe640 1 -- 172.21.15.110:0/2233333153 learned_addr learned my addr 172.21.15.110:0/2233333153 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947daff640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94780ffa50 0x7f9478197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.374+0000 7f947e300640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94781033e0 0x7f947819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947d2fe640 1 -- 172.21.15.110:0/2233333153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94781033e0 msgr2=0x7f947819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947d2fe640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94781033e0 0x7f947819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947d2fe640 1 -- 172.21.15.110:0/2233333153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94780ffa50 msgr2=0x7f9478197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947d2fe640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94780ffa50 0x7f9478197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947d2fe640 1 -- 172.21.15.110:0/2233333153 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947819f590 con 0x7f9478102a20 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947daff640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94780ffa50 0x7f9478197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947d2fe640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9478102a20 0x7f9478197eb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f947400c920 tx=0x7f947400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f946effd640 1 -- 172.21.15.110:0/2233333153 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474007c70 con 0x7f9478102a20 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947819f880 con 0x7f9478102a20 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f946effd640 1 -- 172.21.15.110:0/2233333153 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9474007e10 con 0x7f9478102a20 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f946effd640 1 -- 172.21.15.110:0/2233333153 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474018820 con 0x7f9478102a20 2024-04-25T00:00:10.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.375+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947819fe20 con 0x7f9478102a20 2024-04-25T00:00:10.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.376+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9478108bd0 con 0x7f9478102a20 2024-04-25T00:00:10.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.377+0000 7f946effd640 1 -- 172.21.15.110:0/2233333153 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f94740189c0 con 0x7f9478102a20 2024-04-25T00:00:10.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.378+0000 7f946effd640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f94640a2c20 0x7f94640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:10.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.378+0000 7f947daff640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f94640a2c20 0x7f94640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:10.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.378+0000 7f946effd640 1 -- 172.21.15.110:0/2233333153 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f94740cf240 con 0x7f9478102a20 2024-04-25T00:00:10.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.378+0000 7f947daff640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f94640a2c20 0x7f94640a50e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9468006010 tx=0x7f9468031780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:10.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.382+0000 7f946effd640 1 -- 172.21.15.110:0/2233333153 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9474014030 con 0x7f9478102a20 2024-04-25T00:00:10.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.558+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9478061b80 con 0x7f94640a2c20 2024-04-25T00:00:10.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.565+0000 7f946effd640 1 -- 172.21.15.110:0/2233333153 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258934 (secure 0 0 0) 0x7f9478061b80 con 0x7f94640a2c20 2024-04-25T00:00:10.568 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:10.569 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-04-25T00:00:10.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f94640a2c20 msgr2=0x7f94640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f94640a2c20 0x7f94640a50e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9468006010 tx=0x7f9468031780 comp rx=0 tx=0).stop 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9478102a20 msgr2=0x7f9478197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9478102a20 0x7f9478197eb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f947400c920 tx=0x7f947400cde0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 shutdown_connections 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f94640a2c20 0x7f94640a50e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94781033e0 0x7f947819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9478102a20 0x7f9478197eb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 --2- 172.21.15.110:0/2233333153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94780ffa50 0x7f9478197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 >> 172.21.15.110:0/2233333153 conn(0x7f94780fb380 msgr2=0x7f9478105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 shutdown_connections 2024-04-25T00:00:10.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:10.568+0000 7f947fd63640 1 -- 172.21.15.110:0/2233333153 wait complete. 2024-04-25T00:00:10.580 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-25T00:00:10.508360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":137,"num_read_kb":119,"num_write":297,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":261,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351908,"kb_used_data":11052,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589916,"statfs":{"total":1151940427776,"available":1151580073984,"internally_reserved":0,"allocated":11317248,"data_stored":7047470,"data_compressed":108435,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":116,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.999643"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.670175+0000","last_change":"2024-04-25T00:00:03.143150+0000","last_active":"2024-04-25T00:00:04.670175+0000","last_peered":"2024-04-25T00:00:04.670175+0000","last_clean":"2024-04-25T00:00:04.670175+0000","last_became_active":"2024-04-25T00:00:03.142344+0000","last_became_peered":"2024-04-25T00:00:03.142344+0000","last_unstale":"2024-04-25T00:00:04.670175+0000","last_undegraded":"2024-04-25T00:00:04.670175+0000","last_fullsized":"2024-04-25T00:00:04.670175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:18.293748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239163+0000","last_change":"2024-04-25T00:00:03.145454+0000","last_active":"2024-04-25T00:00:04.239163+0000","last_peered":"2024-04-25T00:00:04.239163+0000","last_clean":"2024-04-25T00:00:04.239163+0000","last_became_active":"2024-04-25T00:00:03.144905+0000","last_became_peered":"2024-04-25T00:00:03.144905+0000","last_unstale":"2024-04-25T00:00:04.239163+0000","last_undegraded":"2024-04-25T00:00:04.239163+0000","last_fullsized":"2024-04-25T00:00:04.239163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:39.583313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239000+0000","last_change":"2024-04-25T00:00:03.141770+0000","last_active":"2024-04-25T00:00:04.239000+0000","last_peered":"2024-04-25T00:00:04.239000+0000","last_clean":"2024-04-25T00:00:04.239000+0000","last_became_active":"2024-04-25T00:00:03.141468+0000","last_became_peered":"2024-04-25T00:00:03.141468+0000","last_unstale":"2024-04-25T00:00:04.239000+0000","last_undegraded":"2024-04-25T00:00:04.239000+0000","last_fullsized":"2024-04-25T00:00:04.239000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:33.112112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240201+0000","last_change":"2024-04-25T00:00:03.142992+0000","last_active":"2024-04-25T00:00:04.240201+0000","last_peered":"2024-04-25T00:00:04.240201+0000","last_clean":"2024-04-25T00:00:04.240201+0000","last_became_active":"2024-04-25T00:00:03.142692+0000","last_became_peered":"2024-04-25T00:00:03.142692+0000","last_unstale":"2024-04-25T00:00:04.240201+0000","last_undegraded":"2024-04-25T00:00:04.240201+0000","last_fullsized":"2024-04-25T00:00:04.240201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:21.513270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239564+0000","last_change":"2024-04-25T00:00:03.146405+0000","last_active":"2024-04-25T00:00:04.239564+0000","last_peered":"2024-04-25T00:00:04.239564+0000","last_clean":"2024-04-25T00:00:04.239564+0000","last_became_active":"2024-04-25T00:00:03.145520+0000","last_became_peered":"2024-04-25T00:00:03.145520+0000","last_unstale":"2024-04-25T00:00:04.239564+0000","last_undegraded":"2024-04-25T00:00:04.239564+0000","last_fullsized":"2024-04-25T00:00:04.239564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:02.256905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128744+0000","last_change":"2024-04-25T00:00:03.142973+0000","last_active":"2024-04-25T00:00:04.128744+0000","last_peered":"2024-04-25T00:00:04.128744+0000","last_clean":"2024-04-25T00:00:04.128744+0000","last_became_active":"2024-04-25T00:00:03.142091+0000","last_became_peered":"2024-04-25T00:00:03.142091+0000","last_unstale":"2024-04-25T00:00:04.128744+0000","last_undegraded":"2024-04-25T00:00:04.128744+0000","last_fullsized":"2024-04-25T00:00:04.128744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:01.948299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239151+0000","last_change":"2024-04-25T00:00:03.148383+0000","last_active":"2024-04-25T00:00:04.239151+0000","last_peered":"2024-04-25T00:00:04.239151+0000","last_clean":"2024-04-25T00:00:04.239151+0000","last_became_active":"2024-04-25T00:00:03.148105+0000","last_became_peered":"2024-04-25T00:00:03.148105+0000","last_unstale":"2024-04-25T00:00:04.239151+0000","last_undegraded":"2024-04-25T00:00:04.239151+0000","last_fullsized":"2024-04-25T00:00:04.239151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:31.902600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238989+0000","last_change":"2024-04-25T00:00:03.151575+0000","last_active":"2024-04-25T00:00:04.238989+0000","last_peered":"2024-04-25T00:00:04.238989+0000","last_clean":"2024-04-25T00:00:04.238989+0000","last_became_active":"2024-04-25T00:00:03.151269+0000","last_became_peered":"2024-04-25T00:00:03.151269+0000","last_unstale":"2024-04-25T00:00:04.238989+0000","last_undegraded":"2024-04-25T00:00:04.238989+0000","last_fullsized":"2024-04-25T00:00:04.238989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:16.561180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239396+0000","last_change":"2024-04-25T00:00:03.140175+0000","last_active":"2024-04-25T00:00:04.239396+0000","last_peered":"2024-04-25T00:00:04.239396+0000","last_clean":"2024-04-25T00:00:04.239396+0000","last_became_active":"2024-04-25T00:00:03.139838+0000","last_became_peered":"2024-04-25T00:00:03.139838+0000","last_unstale":"2024-04-25T00:00:04.239396+0000","last_undegraded":"2024-04-25T00:00:04.239396+0000","last_fullsized":"2024-04-25T00:00:04.239396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:16.768025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.129090+0000","last_change":"2024-04-25T00:00:03.143149+0000","last_active":"2024-04-25T00:00:04.129090+0000","last_peered":"2024-04-25T00:00:04.129090+0000","last_clean":"2024-04-25T00:00:04.129090+0000","last_became_active":"2024-04-25T00:00:03.142799+0000","last_became_peered":"2024-04-25T00:00:03.142799+0000","last_unstale":"2024-04-25T00:00:04.129090+0000","last_undegraded":"2024-04-25T00:00:04.129090+0000","last_fullsized":"2024-04-25T00:00:04.129090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:38.941329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240368+0000","last_change":"2024-04-25T00:00:03.141519+0000","last_active":"2024-04-25T00:00:04.240368+0000","last_peered":"2024-04-25T00:00:04.240368+0000","last_clean":"2024-04-25T00:00:04.240368+0000","last_became_active":"2024-04-25T00:00:03.140452+0000","last_became_peered":"2024-04-25T00:00:03.140452+0000","last_unstale":"2024-04-25T00:00:04.240368+0000","last_undegraded":"2024-04-25T00:00:04.240368+0000","last_fullsized":"2024-04-25T00:00:04.240368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:17.552403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240071+0000","last_change":"2024-04-25T00:00:03.149727+0000","last_active":"2024-04-25T00:00:04.240071+0000","last_peered":"2024-04-25T00:00:04.240071+0000","last_clean":"2024-04-25T00:00:04.240071+0000","last_became_active":"2024-04-25T00:00:03.149205+0000","last_became_peered":"2024-04-25T00:00:03.149205+0000","last_unstale":"2024-04-25T00:00:04.240071+0000","last_undegraded":"2024-04-25T00:00:04.240071+0000","last_fullsized":"2024-04-25T00:00:04.240071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:55.782750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127848+0000","last_change":"2024-04-25T00:00:03.139219+0000","last_active":"2024-04-25T00:00:04.127848+0000","last_peered":"2024-04-25T00:00:04.127848+0000","last_clean":"2024-04-25T00:00:04.127848+0000","last_became_active":"2024-04-25T00:00:03.138883+0000","last_became_peered":"2024-04-25T00:00:03.138883+0000","last_unstale":"2024-04-25T00:00:04.127848+0000","last_undegraded":"2024-04-25T00:00:04.127848+0000","last_fullsized":"2024-04-25T00:00:04.127848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.744932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238771+0000","last_change":"2024-04-25T00:00:03.144473+0000","last_active":"2024-04-25T00:00:04.238771+0000","last_peered":"2024-04-25T00:00:04.238771+0000","last_clean":"2024-04-25T00:00:04.238771+0000","last_became_active":"2024-04-25T00:00:03.144166+0000","last_became_peered":"2024-04-25T00:00:03.144166+0000","last_unstale":"2024-04-25T00:00:04.238771+0000","last_undegraded":"2024-04-25T00:00:04.238771+0000","last_fullsized":"2024-04-25T00:00:04.238771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:25.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238482+0000","last_change":"2024-04-25T00:00:03.143292+0000","last_active":"2024-04-25T00:00:04.238482+0000","last_peered":"2024-04-25T00:00:04.238482+0000","last_clean":"2024-04-25T00:00:04.238482+0000","last_became_active":"2024-04-25T00:00:03.142993+0000","last_became_peered":"2024-04-25T00:00:03.142993+0000","last_unstale":"2024-04-25T00:00:04.238482+0000","last_undegraded":"2024-04-25T00:00:04.238482+0000","last_fullsized":"2024-04-25T00:00:04.238482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.841015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239242+0000","last_change":"2024-04-25T00:00:03.148117+0000","last_active":"2024-04-25T00:00:04.239242+0000","last_peered":"2024-04-25T00:00:04.239242+0000","last_clean":"2024-04-25T00:00:04.239242+0000","last_became_active":"2024-04-25T00:00:03.147670+0000","last_became_peered":"2024-04-25T00:00:03.147670+0000","last_unstale":"2024-04-25T00:00:04.239242+0000","last_undegraded":"2024-04-25T00:00:04.239242+0000","last_fullsized":"2024-04-25T00:00:04.239242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:01.716253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.670331+0000","last_change":"2024-04-25T00:00:03.143452+0000","last_active":"2024-04-25T00:00:04.670331+0000","last_peered":"2024-04-25T00:00:04.670331+0000","last_clean":"2024-04-25T00:00:04.670331+0000","last_became_active":"2024-04-25T00:00:03.142379+0000","last_became_peered":"2024-04-25T00:00:03.142379+0000","last_unstale":"2024-04-25T00:00:04.670331+0000","last_undegraded":"2024-04-25T00:00:04.670331+0000","last_fullsized":"2024-04-25T00:00:04.670331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:46.828363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239367+0000","last_change":"2024-04-25T00:00:03.138570+0000","last_active":"2024-04-25T00:00:04.239367+0000","last_peered":"2024-04-25T00:00:04.239367+0000","last_clean":"2024-04-25T00:00:04.239367+0000","last_became_active":"2024-04-25T00:00:03.138198+0000","last_became_peered":"2024-04-25T00:00:03.138198+0000","last_unstale":"2024-04-25T00:00:04.239367+0000","last_undegraded":"2024-04-25T00:00:04.239367+0000","last_fullsized":"2024-04-25T00:00:04.239367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:47.794135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.129133+0000","last_change":"2024-04-25T00:00:03.143152+0000","last_active":"2024-04-25T00:00:04.129133+0000","last_peered":"2024-04-25T00:00:04.129133+0000","last_clean":"2024-04-25T00:00:04.129133+0000","last_became_active":"2024-04-25T00:00:03.142812+0000","last_became_peered":"2024-04-25T00:00:03.142812+0000","last_unstale":"2024-04-25T00:00:04.129133+0000","last_undegraded":"2024-04-25T00:00:04.129133+0000","last_fullsized":"2024-04-25T00:00:04.129133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:58.808783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238607+0000","last_change":"2024-04-25T00:00:02.525740+0000","last_active":"2024-04-25T00:00:04.238607+0000","last_peered":"2024-04-25T00:00:04.238607+0000","last_clean":"2024-04-25T00:00:04.238607+0000","last_became_active":"2024-04-25T00:00:02.525429+0000","last_became_peered":"2024-04-25T00:00:02.525429+0000","last_unstale":"2024-04-25T00:00:04.238607+0000","last_undegraded":"2024-04-25T00:00:04.238607+0000","last_fullsized":"2024-04-25T00:00:04.238607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:43.585678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239381+0000","last_change":"2024-04-25T00:00:03.151963+0000","last_active":"2024-04-25T00:00:04.239381+0000","last_peered":"2024-04-25T00:00:04.239381+0000","last_clean":"2024-04-25T00:00:04.239381+0000","last_became_active":"2024-04-25T00:00:03.151059+0000","last_became_peered":"2024-04-25T00:00:03.151059+0000","last_unstale":"2024-04-25T00:00:04.239381+0000","last_undegraded":"2024-04-25T00:00:04.239381+0000","last_fullsized":"2024-04-25T00:00:04.239381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:37.834035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239548+0000","last_change":"2024-04-25T00:00:02.534181+0000","last_active":"2024-04-25T00:00:04.239548+0000","last_peered":"2024-04-25T00:00:04.239548+0000","last_clean":"2024-04-25T00:00:04.239548+0000","last_became_active":"2024-04-25T00:00:02.533848+0000","last_became_peered":"2024-04-25T00:00:02.533848+0000","last_unstale":"2024-04-25T00:00:04.239548+0000","last_undegraded":"2024-04-25T00:00:04.239548+0000","last_fullsized":"2024-04-25T00:00:04.239548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:25.276477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239356+0000","last_change":"2024-04-25T00:00:03.142038+0000","last_active":"2024-04-25T00:00:04.239356+0000","last_peered":"2024-04-25T00:00:04.239356+0000","last_clean":"2024-04-25T00:00:04.239356+0000","last_became_active":"2024-04-25T00:00:03.141705+0000","last_became_peered":"2024-04-25T00:00:03.141705+0000","last_unstale":"2024-04-25T00:00:04.239356+0000","last_undegraded":"2024-04-25T00:00:04.239356+0000","last_fullsized":"2024-04-25T00:00:04.239356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.382031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.557607+0000","last_change":"2024-04-25T00:00:02.526339+0000","last_active":"2024-04-25T00:00:06.557607+0000","last_peered":"2024-04-25T00:00:06.557607+0000","last_clean":"2024-04-25T00:00:06.557607+0000","last_became_active":"2024-04-25T00:00:02.525999+0000","last_became_peered":"2024-04-25T00:00:02.525999+0000","last_unstale":"2024-04-25T00:00:06.557607+0000","last_undegraded":"2024-04-25T00:00:06.557607+0000","last_fullsized":"2024-04-25T00:00:06.557607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:52.997821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238676+0000","last_change":"2024-04-25T00:00:03.143896+0000","last_active":"2024-04-25T00:00:04.238676+0000","last_peered":"2024-04-25T00:00:04.238676+0000","last_clean":"2024-04-25T00:00:04.238676+0000","last_became_active":"2024-04-25T00:00:03.141311+0000","last_became_peered":"2024-04-25T00:00:03.141311+0000","last_unstale":"2024-04-25T00:00:04.238676+0000","last_undegraded":"2024-04-25T00:00:04.238676+0000","last_fullsized":"2024-04-25T00:00:04.238676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:57.819319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239855+0000","last_change":"2024-04-25T00:00:02.523497+0000","last_active":"2024-04-25T00:00:04.239855+0000","last_peered":"2024-04-25T00:00:04.239855+0000","last_clean":"2024-04-25T00:00:04.239855+0000","last_became_active":"2024-04-25T00:00:02.523153+0000","last_became_peered":"2024-04-25T00:00:02.523153+0000","last_unstale":"2024-04-25T00:00:04.239855+0000","last_undegraded":"2024-04-25T00:00:04.239855+0000","last_fullsized":"2024-04-25T00:00:04.239855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:44.473230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239507+0000","last_change":"2024-04-25T00:00:03.140867+0000","last_active":"2024-04-25T00:00:04.239507+0000","last_peered":"2024-04-25T00:00:04.239507+0000","last_clean":"2024-04-25T00:00:04.239507+0000","last_became_active":"2024-04-25T00:00:03.140545+0000","last_became_peered":"2024-04-25T00:00:03.140545+0000","last_unstale":"2024-04-25T00:00:04.239507+0000","last_undegraded":"2024-04-25T00:00:04.239507+0000","last_fullsized":"2024-04-25T00:00:04.239507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:46.928359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554801+0000","last_change":"2024-04-25T00:00:02.528897+0000","last_active":"2024-04-25T00:00:05.554801+0000","last_peered":"2024-04-25T00:00:05.554801+0000","last_clean":"2024-04-25T00:00:05.554801+0000","last_became_active":"2024-04-25T00:00:02.528584+0000","last_became_peered":"2024-04-25T00:00:02.528584+0000","last_unstale":"2024-04-25T00:00:05.554801+0000","last_undegraded":"2024-04-25T00:00:05.554801+0000","last_fullsized":"2024-04-25T00:00:05.554801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:23.437956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238669+0000","last_change":"2024-04-25T00:00:03.147692+0000","last_active":"2024-04-25T00:00:04.238669+0000","last_peered":"2024-04-25T00:00:04.238669+0000","last_clean":"2024-04-25T00:00:04.238669+0000","last_became_active":"2024-04-25T00:00:03.147183+0000","last_became_peered":"2024-04-25T00:00:03.147183+0000","last_unstale":"2024-04-25T00:00:04.238669+0000","last_undegraded":"2024-04-25T00:00:04.238669+0000","last_fullsized":"2024-04-25T00:00:04.238669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:12.640665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.559274+0000","last_change":"2024-04-25T00:00:02.522387+0000","last_active":"2024-04-25T00:00:06.559274+0000","last_peered":"2024-04-25T00:00:06.559274+0000","last_clean":"2024-04-25T00:00:06.559274+0000","last_became_active":"2024-04-25T00:00:02.522093+0000","last_became_peered":"2024-04-25T00:00:02.522093+0000","last_unstale":"2024-04-25T00:00:06.559274+0000","last_undegraded":"2024-04-25T00:00:06.559274+0000","last_fullsized":"2024-04-25T00:00:06.559274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:33.393915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239511+0000","last_change":"2024-04-25T00:00:03.146277+0000","last_active":"2024-04-25T00:00:04.239511+0000","last_peered":"2024-04-25T00:00:04.239511+0000","last_clean":"2024-04-25T00:00:04.239511+0000","last_became_active":"2024-04-25T00:00:03.145846+0000","last_became_peered":"2024-04-25T00:00:03.145846+0000","last_unstale":"2024-04-25T00:00:04.239511+0000","last_undegraded":"2024-04-25T00:00:04.239511+0000","last_fullsized":"2024-04-25T00:00:04.239511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.692246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.558430+0000","last_change":"2024-04-25T00:00:02.528405+0000","last_active":"2024-04-25T00:00:06.558430+0000","last_peered":"2024-04-25T00:00:06.558430+0000","last_clean":"2024-04-25T00:00:06.558430+0000","last_became_active":"2024-04-25T00:00:02.528115+0000","last_became_peered":"2024-04-25T00:00:02.528115+0000","last_unstale":"2024-04-25T00:00:06.558430+0000","last_undegraded":"2024-04-25T00:00:06.558430+0000","last_fullsized":"2024-04-25T00:00:06.558430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:51.382361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238306+0000","last_change":"2024-04-25T00:00:03.144266+0000","last_active":"2024-04-25T00:00:04.238306+0000","last_peered":"2024-04-25T00:00:04.238306+0000","last_clean":"2024-04-25T00:00:04.238306+0000","last_became_active":"2024-04-25T00:00:03.141704+0000","last_became_peered":"2024-04-25T00:00:03.141704+0000","last_unstale":"2024-04-25T00:00:04.238306+0000","last_undegraded":"2024-04-25T00:00:04.238306+0000","last_fullsized":"2024-04-25T00:00:04.238306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:14.161783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238781+0000","last_change":"2024-04-25T00:00:02.528134+0000","last_active":"2024-04-25T00:00:04.238781+0000","last_peered":"2024-04-25T00:00:04.238781+0000","last_clean":"2024-04-25T00:00:04.238781+0000","last_became_active":"2024-04-25T00:00:02.527830+0000","last_became_peered":"2024-04-25T00:00:02.527830+0000","last_unstale":"2024-04-25T00:00:04.238781+0000","last_undegraded":"2024-04-25T00:00:04.238781+0000","last_fullsized":"2024-04-25T00:00:04.238781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:46.846244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239053+0000","last_change":"2024-04-25T00:00:03.143505+0000","last_active":"2024-04-25T00:00:04.239053+0000","last_peered":"2024-04-25T00:00:04.239053+0000","last_clean":"2024-04-25T00:00:04.239053+0000","last_became_active":"2024-04-25T00:00:03.143131+0000","last_became_peered":"2024-04-25T00:00:03.143131+0000","last_unstale":"2024-04-25T00:00:04.239053+0000","last_undegraded":"2024-04-25T00:00:04.239053+0000","last_fullsized":"2024-04-25T00:00:04.239053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:23.743162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554221+0000","last_change":"2024-04-25T00:00:02.525978+0000","last_active":"2024-04-25T00:00:05.554221+0000","last_peered":"2024-04-25T00:00:05.554221+0000","last_clean":"2024-04-25T00:00:05.554221+0000","last_became_active":"2024-04-25T00:00:02.525641+0000","last_became_peered":"2024-04-25T00:00:02.525641+0000","last_unstale":"2024-04-25T00:00:05.554221+0000","last_undegraded":"2024-04-25T00:00:05.554221+0000","last_fullsized":"2024-04-25T00:00:05.554221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:58.508573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238917+0000","last_change":"2024-04-25T00:00:03.143734+0000","last_active":"2024-04-25T00:00:04.238917+0000","last_peered":"2024-04-25T00:00:04.238917+0000","last_clean":"2024-04-25T00:00:04.238917+0000","last_became_active":"2024-04-25T00:00:03.143402+0000","last_became_peered":"2024-04-25T00:00:03.143402+0000","last_unstale":"2024-04-25T00:00:04.238917+0000","last_undegraded":"2024-04-25T00:00:04.238917+0000","last_fullsized":"2024-04-25T00:00:04.238917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:36.608803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.553889+0000","last_change":"2024-04-25T00:00:02.530567+0000","last_active":"2024-04-25T00:00:05.553889+0000","last_peered":"2024-04-25T00:00:05.553889+0000","last_clean":"2024-04-25T00:00:05.553889+0000","last_became_active":"2024-04-25T00:00:02.530066+0000","last_became_peered":"2024-04-25T00:00:02.530066+0000","last_unstale":"2024-04-25T00:00:05.553889+0000","last_undegraded":"2024-04-25T00:00:05.553889+0000","last_fullsized":"2024-04-25T00:00:05.553889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:19.892052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239299+0000","last_change":"2024-04-25T00:00:03.146905+0000","last_active":"2024-04-25T00:00:04.239299+0000","last_peered":"2024-04-25T00:00:04.239299+0000","last_clean":"2024-04-25T00:00:04.239299+0000","last_became_active":"2024-04-25T00:00:03.146080+0000","last_became_peered":"2024-04-25T00:00:03.146080+0000","last_unstale":"2024-04-25T00:00:04.239299+0000","last_undegraded":"2024-04-25T00:00:04.239299+0000","last_fullsized":"2024-04-25T00:00:04.239299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:42.764082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239677+0000","last_change":"2024-04-25T00:00:02.527667+0000","last_active":"2024-04-25T00:00:04.239677+0000","last_peered":"2024-04-25T00:00:04.239677+0000","last_clean":"2024-04-25T00:00:04.239677+0000","last_became_active":"2024-04-25T00:00:02.527333+0000","last_became_peered":"2024-04-25T00:00:02.527333+0000","last_unstale":"2024-04-25T00:00:04.239677+0000","last_undegraded":"2024-04-25T00:00:04.239677+0000","last_fullsized":"2024-04-25T00:00:04.239677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:15.982704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239060+0000","last_change":"2024-04-25T00:00:03.145856+0000","last_active":"2024-04-25T00:00:04.239060+0000","last_peered":"2024-04-25T00:00:04.239060+0000","last_clean":"2024-04-25T00:00:04.239060+0000","last_became_active":"2024-04-25T00:00:03.145406+0000","last_became_peered":"2024-04-25T00:00:03.145406+0000","last_unstale":"2024-04-25T00:00:04.239060+0000","last_undegraded":"2024-04-25T00:00:04.239060+0000","last_fullsized":"2024-04-25T00:00:04.239060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:28.318575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:07.561685+0000","last_change":"2024-04-25T00:00:02.522294+0000","last_active":"2024-04-25T00:00:07.561685+0000","last_peered":"2024-04-25T00:00:07.561685+0000","last_clean":"2024-04-25T00:00:07.561685+0000","last_became_active":"2024-04-25T00:00:02.521776+0000","last_became_peered":"2024-04-25T00:00:02.521776+0000","last_unstale":"2024-04-25T00:00:07.561685+0000","last_undegraded":"2024-04-25T00:00:07.561685+0000","last_fullsized":"2024-04-25T00:00:07.561685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:07.677118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238875+0000","last_change":"2024-04-25T00:00:03.143877+0000","last_active":"2024-04-25T00:00:04.238875+0000","last_peered":"2024-04-25T00:00:04.238875+0000","last_clean":"2024-04-25T00:00:04.238875+0000","last_became_active":"2024-04-25T00:00:03.143558+0000","last_became_peered":"2024-04-25T00:00:03.143558+0000","last_unstale":"2024-04-25T00:00:04.238875+0000","last_undegraded":"2024-04-25T00:00:04.238875+0000","last_fullsized":"2024-04-25T00:00:04.238875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:37.306175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.557497+0000","last_change":"2024-04-25T00:00:02.520760+0000","last_active":"2024-04-25T00:00:06.557497+0000","last_peered":"2024-04-25T00:00:06.557497+0000","last_clean":"2024-04-25T00:00:06.557497+0000","last_became_active":"2024-04-25T00:00:02.520417+0000","last_became_peered":"2024-04-25T00:00:02.520417+0000","last_unstale":"2024-04-25T00:00:06.557497+0000","last_undegraded":"2024-04-25T00:00:06.557497+0000","last_fullsized":"2024-04-25T00:00:06.557497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:53.895653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127633+0000","last_change":"2024-04-25T00:00:03.144157+0000","last_active":"2024-04-25T00:00:04.127633+0000","last_peered":"2024-04-25T00:00:04.127633+0000","last_clean":"2024-04-25T00:00:04.127633+0000","last_became_active":"2024-04-25T00:00:03.143816+0000","last_became_peered":"2024-04-25T00:00:03.143816+0000","last_unstale":"2024-04-25T00:00:04.127633+0000","last_undegraded":"2024-04-25T00:00:04.127633+0000","last_fullsized":"2024-04-25T00:00:04.127633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:45.007842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554504+0000","last_change":"2024-04-25T00:00:02.534393+0000","last_active":"2024-04-25T00:00:05.554504+0000","last_peered":"2024-04-25T00:00:05.554504+0000","last_clean":"2024-04-25T00:00:05.554504+0000","last_became_active":"2024-04-25T00:00:02.534045+0000","last_became_peered":"2024-04-25T00:00:02.534045+0000","last_unstale":"2024-04-25T00:00:05.554504+0000","last_undegraded":"2024-04-25T00:00:05.554504+0000","last_fullsized":"2024-04-25T00:00:05.554504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:35.547065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240074+0000","last_change":"2024-04-25T00:00:03.149787+0000","last_active":"2024-04-25T00:00:04.240074+0000","last_peered":"2024-04-25T00:00:04.240074+0000","last_clean":"2024-04-25T00:00:04.240074+0000","last_became_active":"2024-04-25T00:00:03.149482+0000","last_became_peered":"2024-04-25T00:00:03.149482+0000","last_unstale":"2024-04-25T00:00:04.240074+0000","last_undegraded":"2024-04-25T00:00:04.240074+0000","last_fullsized":"2024-04-25T00:00:04.240074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:32.278190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.558252+0000","last_change":"2024-04-25T00:00:02.520357+0000","last_active":"2024-04-25T00:00:06.558252+0000","last_peered":"2024-04-25T00:00:06.558252+0000","last_clean":"2024-04-25T00:00:06.558252+0000","last_became_active":"2024-04-25T00:00:02.519926+0000","last_became_peered":"2024-04-25T00:00:02.519926+0000","last_unstale":"2024-04-25T00:00:06.558252+0000","last_undegraded":"2024-04-25T00:00:06.558252+0000","last_fullsized":"2024-04-25T00:00:06.558252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:52.438584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240147+0000","last_change":"2024-04-25T00:00:03.147560+0000","last_active":"2024-04-25T00:00:04.240147+0000","last_peered":"2024-04-25T00:00:04.240147+0000","last_clean":"2024-04-25T00:00:04.240147+0000","last_became_active":"2024-04-25T00:00:03.147234+0000","last_became_peered":"2024-04-25T00:00:03.147234+0000","last_unstale":"2024-04-25T00:00:04.240147+0000","last_undegraded":"2024-04-25T00:00:04.240147+0000","last_fullsized":"2024-04-25T00:00:04.240147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:33.737763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.560092+0000","last_change":"2024-04-25T00:00:02.528614+0000","last_active":"2024-04-25T00:00:06.560092+0000","last_peered":"2024-04-25T00:00:06.560092+0000","last_clean":"2024-04-25T00:00:06.560092+0000","last_became_active":"2024-04-25T00:00:02.528203+0000","last_became_peered":"2024-04-25T00:00:02.528203+0000","last_unstale":"2024-04-25T00:00:06.560092+0000","last_undegraded":"2024-04-25T00:00:06.560092+0000","last_fullsized":"2024-04-25T00:00:06.560092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:33.116699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239244+0000","last_change":"2024-04-25T00:00:03.145009+0000","last_active":"2024-04-25T00:00:04.239244+0000","last_peered":"2024-04-25T00:00:04.239244+0000","last_clean":"2024-04-25T00:00:04.239244+0000","last_became_active":"2024-04-25T00:00:03.144575+0000","last_became_peered":"2024-04-25T00:00:03.144575+0000","last_unstale":"2024-04-25T00:00:04.239244+0000","last_undegraded":"2024-04-25T00:00:04.239244+0000","last_fullsized":"2024-04-25T00:00:04.239244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.219880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:07.554167+0000","last_change":"2024-04-25T00:00:02.524951+0000","last_active":"2024-04-25T00:00:07.554167+0000","last_peered":"2024-04-25T00:00:07.554167+0000","last_clean":"2024-04-25T00:00:07.554167+0000","last_became_active":"2024-04-25T00:00:02.524624+0000","last_became_peered":"2024-04-25T00:00:02.524624+0000","last_unstale":"2024-04-25T00:00:07.554167+0000","last_undegraded":"2024-04-25T00:00:07.554167+0000","last_fullsized":"2024-04-25T00:00:07.554167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:23.381521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.670262+0000","last_change":"2024-04-25T00:00:03.145446+0000","last_active":"2024-04-25T00:00:04.670262+0000","last_peered":"2024-04-25T00:00:04.670262+0000","last_clean":"2024-04-25T00:00:04.670262+0000","last_became_active":"2024-04-25T00:00:03.145125+0000","last_became_peered":"2024-04-25T00:00:03.145125+0000","last_unstale":"2024-04-25T00:00:04.670262+0000","last_undegraded":"2024-04-25T00:00:04.670262+0000","last_fullsized":"2024-04-25T00:00:04.670262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:40.187270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238725+0000","last_change":"2024-04-25T00:00:02.527499+0000","last_active":"2024-04-25T00:00:04.238725+0000","last_peered":"2024-04-25T00:00:04.238725+0000","last_clean":"2024-04-25T00:00:04.238725+0000","last_became_active":"2024-04-25T00:00:02.527151+0000","last_became_peered":"2024-04-25T00:00:02.527151+0000","last_unstale":"2024-04-25T00:00:04.238725+0000","last_undegraded":"2024-04-25T00:00:04.238725+0000","last_fullsized":"2024-04-25T00:00:04.238725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:09.013930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.129109+0000","last_change":"2024-04-25T00:00:03.148231+0000","last_active":"2024-04-25T00:00:04.129109+0000","last_peered":"2024-04-25T00:00:04.129109+0000","last_clean":"2024-04-25T00:00:04.129109+0000","last_became_active":"2024-04-25T00:00:03.147891+0000","last_became_peered":"2024-04-25T00:00:03.147891+0000","last_unstale":"2024-04-25T00:00:04.129109+0000","last_undegraded":"2024-04-25T00:00:04.129109+0000","last_fullsized":"2024-04-25T00:00:04.129109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:42.784699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240238+0000","last_change":"2024-04-25T00:00:02.528234+0000","last_active":"2024-04-25T00:00:04.240238+0000","last_peered":"2024-04-25T00:00:04.240238+0000","last_clean":"2024-04-25T00:00:04.240238+0000","last_became_active":"2024-04-25T00:00:02.527875+0000","last_became_peered":"2024-04-25T00:00:02.527875+0000","last_unstale":"2024-04-25T00:00:04.240238+0000","last_undegraded":"2024-04-25T00:00:04.240238+0000","last_fullsized":"2024-04-25T00:00:04.240238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:06.634497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128888+0000","last_change":"2024-04-25T00:00:03.144246+0000","last_active":"2024-04-25T00:00:04.128888+0000","last_peered":"2024-04-25T00:00:04.128888+0000","last_clean":"2024-04-25T00:00:04.128888+0000","last_became_active":"2024-04-25T00:00:03.143925+0000","last_became_peered":"2024-04-25T00:00:03.143925+0000","last_unstale":"2024-04-25T00:00:04.128888+0000","last_undegraded":"2024-04-25T00:00:04.128888+0000","last_fullsized":"2024-04-25T00:00:04.128888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:51.741275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239124+0000","last_change":"2024-04-25T00:00:03.141261+0000","last_active":"2024-04-25T00:00:04.239124+0000","last_peered":"2024-04-25T00:00:04.239124+0000","last_clean":"2024-04-25T00:00:04.239124+0000","last_became_active":"2024-04-25T00:00:03.137943+0000","last_became_peered":"2024-04-25T00:00:03.137943+0000","last_unstale":"2024-04-25T00:00:04.239124+0000","last_undegraded":"2024-04-25T00:00:04.239124+0000","last_fullsized":"2024-04-25T00:00:04.239124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:41.655749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.559557+0000","last_change":"2024-04-25T00:00:02.522048+0000","last_active":"2024-04-25T00:00:06.559557+0000","last_peered":"2024-04-25T00:00:06.559557+0000","last_clean":"2024-04-25T00:00:06.559557+0000","last_became_active":"2024-04-25T00:00:02.521755+0000","last_became_peered":"2024-04-25T00:00:02.521755+0000","last_unstale":"2024-04-25T00:00:06.559557+0000","last_undegraded":"2024-04-25T00:00:06.559557+0000","last_fullsized":"2024-04-25T00:00:06.559557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:58.602357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239320+0000","last_change":"2024-04-25T00:00:03.151428+0000","last_active":"2024-04-25T00:00:04.239320+0000","last_peered":"2024-04-25T00:00:04.239320+0000","last_clean":"2024-04-25T00:00:04.239320+0000","last_became_active":"2024-04-25T00:00:03.151005+0000","last_became_peered":"2024-04-25T00:00:03.151005+0000","last_unstale":"2024-04-25T00:00:04.239320+0000","last_undegraded":"2024-04-25T00:00:04.239320+0000","last_fullsized":"2024-04-25T00:00:04.239320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:28.044803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554997+0000","last_change":"2024-04-25T00:00:02.515714+0000","last_active":"2024-04-25T00:00:05.554997+0000","last_peered":"2024-04-25T00:00:05.554997+0000","last_clean":"2024-04-25T00:00:05.554997+0000","last_became_active":"2024-04-25T00:00:02.515373+0000","last_became_peered":"2024-04-25T00:00:02.515373+0000","last_unstale":"2024-04-25T00:00:05.554997+0000","last_undegraded":"2024-04-25T00:00:05.554997+0000","last_fullsized":"2024-04-25T00:00:05.554997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:34.344261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239409+0000","last_change":"2024-04-24T23:58:50.595622+0000","last_active":"2024-04-25T00:00:04.239409+0000","last_peered":"2024-04-25T00:00:04.239409+0000","last_clean":"2024-04-25T00:00:04.239409+0000","last_became_active":"2024-04-24T23:58:50.594971+0000","last_became_peered":"2024-04-24T23:58:50.594971+0000","last_unstale":"2024-04-25T00:00:04.239409+0000","last_undegraded":"2024-04-25T00:00:04.239409+0000","last_fullsized":"2024-04-25T00:00:04.239409+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:55:22.756734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:55:22.756734+0000","last_clean_scrub_stamp":"2024-04-24T23:55:22.756734+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:20.241007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127517+0000","last_change":"2024-04-25T00:00:03.135594+0000","last_active":"2024-04-25T00:00:04.127517+0000","last_peered":"2024-04-25T00:00:04.127517+0000","last_clean":"2024-04-25T00:00:04.127517+0000","last_became_active":"2024-04-25T00:00:03.135279+0000","last_became_peered":"2024-04-25T00:00:03.135279+0000","last_unstale":"2024-04-25T00:00:04.127517+0000","last_undegraded":"2024-04-25T00:00:04.127517+0000","last_fullsized":"2024-04-25T00:00:04.127517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:37.690119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.558627+0000","last_change":"2024-04-25T00:00:02.527294+0000","last_active":"2024-04-25T00:00:06.558627+0000","last_peered":"2024-04-25T00:00:06.558627+0000","last_clean":"2024-04-25T00:00:06.558627+0000","last_became_active":"2024-04-25T00:00:02.526979+0000","last_became_peered":"2024-04-25T00:00:02.526979+0000","last_unstale":"2024-04-25T00:00:06.558627+0000","last_undegraded":"2024-04-25T00:00:06.558627+0000","last_fullsized":"2024-04-25T00:00:06.558627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:04.784065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.552211+0000","last_change":"2024-04-25T00:00:02.527545+0000","last_active":"2024-04-25T00:00:05.552211+0000","last_peered":"2024-04-25T00:00:05.552211+0000","last_clean":"2024-04-25T00:00:05.552211+0000","last_became_active":"2024-04-25T00:00:02.527250+0000","last_became_peered":"2024-04-25T00:00:02.527250+0000","last_unstale":"2024-04-25T00:00:05.552211+0000","last_undegraded":"2024-04-25T00:00:05.552211+0000","last_fullsized":"2024-04-25T00:00:05.552211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:57.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127727+0000","last_change":"2024-04-25T00:00:03.139578+0000","last_active":"2024-04-25T00:00:04.127727+0000","last_peered":"2024-04-25T00:00:04.127727+0000","last_clean":"2024-04-25T00:00:04.127727+0000","last_became_active":"2024-04-25T00:00:03.139128+0000","last_became_peered":"2024-04-25T00:00:03.139128+0000","last_unstale":"2024-04-25T00:00:04.127727+0000","last_undegraded":"2024-04-25T00:00:04.127727+0000","last_fullsized":"2024-04-25T00:00:04.127727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:33.509293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.525292+0000","last_change":"2024-04-25T00:00:02.520616+0000","last_active":"2024-04-25T00:00:05.525292+0000","last_peered":"2024-04-25T00:00:05.525292+0000","last_clean":"2024-04-25T00:00:05.525292+0000","last_became_active":"2024-04-25T00:00:02.519057+0000","last_became_peered":"2024-04-25T00:00:02.519057+0000","last_unstale":"2024-04-25T00:00:05.525292+0000","last_undegraded":"2024-04-25T00:00:05.525292+0000","last_fullsized":"2024-04-25T00:00:05.525292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:25.847055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239078+0000","last_change":"2024-04-25T00:00:03.150010+0000","last_active":"2024-04-25T00:00:04.239078+0000","last_peered":"2024-04-25T00:00:04.239078+0000","last_clean":"2024-04-25T00:00:04.239078+0000","last_became_active":"2024-04-25T00:00:03.149677+0000","last_became_peered":"2024-04-25T00:00:03.149677+0000","last_unstale":"2024-04-25T00:00:04.239078+0000","last_undegraded":"2024-04-25T00:00:04.239078+0000","last_fullsized":"2024-04-25T00:00:04.239078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:18.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.551533+0000","last_change":"2024-04-25T00:00:02.533171+0000","last_active":"2024-04-25T00:00:05.551533+0000","last_peered":"2024-04-25T00:00:05.551533+0000","last_clean":"2024-04-25T00:00:05.551533+0000","last_became_active":"2024-04-25T00:00:02.532887+0000","last_became_peered":"2024-04-25T00:00:02.532887+0000","last_unstale":"2024-04-25T00:00:05.551533+0000","last_undegraded":"2024-04-25T00:00:05.551533+0000","last_fullsized":"2024-04-25T00:00:05.551533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:24.729809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239095+0000","last_change":"2024-04-25T00:00:03.147457+0000","last_active":"2024-04-25T00:00:04.239095+0000","last_peered":"2024-04-25T00:00:04.239095+0000","last_clean":"2024-04-25T00:00:04.239095+0000","last_became_active":"2024-04-25T00:00:03.147163+0000","last_became_peered":"2024-04-25T00:00:03.147163+0000","last_unstale":"2024-04-25T00:00:04.239095+0000","last_undegraded":"2024-04-25T00:00:04.239095+0000","last_fullsized":"2024-04-25T00:00:04.239095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:32.422006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554061+0000","last_change":"2024-04-25T00:00:02.527573+0000","last_active":"2024-04-25T00:00:05.554061+0000","last_peered":"2024-04-25T00:00:05.554061+0000","last_clean":"2024-04-25T00:00:05.554061+0000","last_became_active":"2024-04-25T00:00:02.527267+0000","last_became_peered":"2024-04-25T00:00:02.527267+0000","last_unstale":"2024-04-25T00:00:05.554061+0000","last_undegraded":"2024-04-25T00:00:05.554061+0000","last_fullsized":"2024-04-25T00:00:05.554061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:31.581529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238575+0000","last_change":"2024-04-25T00:00:03.146602+0000","last_active":"2024-04-25T00:00:04.238575+0000","last_peered":"2024-04-25T00:00:04.238575+0000","last_clean":"2024-04-25T00:00:04.238575+0000","last_became_active":"2024-04-25T00:00:03.145480+0000","last_became_peered":"2024-04-25T00:00:03.145480+0000","last_unstale":"2024-04-25T00:00:04.238575+0000","last_undegraded":"2024-04-25T00:00:04.238575+0000","last_fullsized":"2024-04-25T00:00:04.238575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:02.803903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554920+0000","last_change":"2024-04-25T00:00:02.526567+0000","last_active":"2024-04-25T00:00:05.554920+0000","last_peered":"2024-04-25T00:00:05.554920+0000","last_clean":"2024-04-25T00:00:05.554920+0000","last_became_active":"2024-04-25T00:00:02.526234+0000","last_became_peered":"2024-04-25T00:00:02.526234+0000","last_unstale":"2024-04-25T00:00:05.554920+0000","last_undegraded":"2024-04-25T00:00:05.554920+0000","last_fullsized":"2024-04-25T00:00:05.554920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:14.964206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.669900+0000","last_change":"2024-04-25T00:00:03.140959+0000","last_active":"2024-04-25T00:00:04.669900+0000","last_peered":"2024-04-25T00:00:04.669900+0000","last_clean":"2024-04-25T00:00:04.669900+0000","last_became_active":"2024-04-25T00:00:03.140533+0000","last_became_peered":"2024-04-25T00:00:03.140533+0000","last_unstale":"2024-04-25T00:00:04.669900+0000","last_undegraded":"2024-04-25T00:00:04.669900+0000","last_fullsized":"2024-04-25T00:00:04.669900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:46.166051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128803+0000","last_change":"2024-04-25T00:00:02.527362+0000","last_active":"2024-04-25T00:00:04.128803+0000","last_peered":"2024-04-25T00:00:04.128803+0000","last_clean":"2024-04-25T00:00:04.128803+0000","last_became_active":"2024-04-25T00:00:02.527037+0000","last_became_peered":"2024-04-25T00:00:02.527037+0000","last_unstale":"2024-04-25T00:00:04.128803+0000","last_undegraded":"2024-04-25T00:00:04.128803+0000","last_fullsized":"2024-04-25T00:00:04.128803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:53.365117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239070+0000","last_change":"2024-04-25T00:00:03.149740+0000","last_active":"2024-04-25T00:00:04.239070+0000","last_peered":"2024-04-25T00:00:04.239070+0000","last_clean":"2024-04-25T00:00:04.239070+0000","last_became_active":"2024-04-25T00:00:03.149290+0000","last_became_peered":"2024-04-25T00:00:03.149290+0000","last_unstale":"2024-04-25T00:00:04.239070+0000","last_undegraded":"2024-04-25T00:00:04.239070+0000","last_fullsized":"2024-04-25T00:00:04.239070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.716974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:05.554386+0000","last_change":"2024-04-25T00:00:02.526823+0000","last_active":"2024-04-25T00:00:05.554386+0000","last_peered":"2024-04-25T00:00:05.554386+0000","last_clean":"2024-04-25T00:00:05.554386+0000","last_became_active":"2024-04-25T00:00:02.526529+0000","last_became_peered":"2024-04-25T00:00:02.526529+0000","last_unstale":"2024-04-25T00:00:05.554386+0000","last_undegraded":"2024-04-25T00:00:05.554386+0000","last_fullsized":"2024-04-25T00:00:05.554386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:13.907985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240466+0000","last_change":"2024-04-25T00:00:03.141973+0000","last_active":"2024-04-25T00:00:04.240466+0000","last_peered":"2024-04-25T00:00:04.240466+0000","last_clean":"2024-04-25T00:00:04.240466+0000","last_became_active":"2024-04-25T00:00:03.141560+0000","last_became_peered":"2024-04-25T00:00:03.141560+0000","last_unstale":"2024-04-25T00:00:04.240466+0000","last_undegraded":"2024-04-25T00:00:04.240466+0000","last_fullsized":"2024-04-25T00:00:04.240466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:03.002939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:06.560548+0000","last_change":"2024-04-25T00:00:02.533013+0000","last_active":"2024-04-25T00:00:06.560548+0000","last_peered":"2024-04-25T00:00:06.560548+0000","last_clean":"2024-04-25T00:00:06.560548+0000","last_became_active":"2024-04-25T00:00:02.532667+0000","last_became_peered":"2024-04-25T00:00:02.532667+0000","last_unstale":"2024-04-25T00:00:06.560548+0000","last_undegraded":"2024-04-25T00:00:06.560548+0000","last_fullsized":"2024-04-25T00:00:06.560548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:07.469136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240415+0000","last_change":"2024-04-25T00:00:03.140120+0000","last_active":"2024-04-25T00:00:04.240415+0000","last_peered":"2024-04-25T00:00:04.240415+0000","last_clean":"2024-04-25T00:00:04.240415+0000","last_became_active":"2024-04-25T00:00:03.139778+0000","last_became_peered":"2024-04-25T00:00:03.139778+0000","last_unstale":"2024-04-25T00:00:04.240415+0000","last_undegraded":"2024-04-25T00:00:04.240415+0000","last_fullsized":"2024-04-25T00:00:04.240415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:02.678303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239086+0000","last_change":"2024-04-25T00:00:02.534359+0000","last_active":"2024-04-25T00:00:04.239086+0000","last_peered":"2024-04-25T00:00:04.239086+0000","last_clean":"2024-04-25T00:00:04.239086+0000","last_became_active":"2024-04-25T00:00:02.534057+0000","last_became_peered":"2024-04-25T00:00:02.534057+0000","last_unstale":"2024-04-25T00:00:04.239086+0000","last_undegraded":"2024-04-25T00:00:04.239086+0000","last_fullsized":"2024-04-25T00:00:04.239086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:38.716263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239636+0000","last_change":"2024-04-25T00:00:03.146608+0000","last_active":"2024-04-25T00:00:04.239636+0000","last_peered":"2024-04-25T00:00:04.239636+0000","last_clean":"2024-04-25T00:00:04.239636+0000","last_became_active":"2024-04-25T00:00:03.146256+0000","last_became_peered":"2024-04-25T00:00:03.146256+0000","last_unstale":"2024-04-25T00:00:04.239636+0000","last_undegraded":"2024-04-25T00:00:04.239636+0000","last_fullsized":"2024-04-25T00:00:04.239636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:04.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127851+0000","last_change":"2024-04-25T00:00:02.524736+0000","last_active":"2024-04-25T00:00:04.127851+0000","last_peered":"2024-04-25T00:00:04.127851+0000","last_clean":"2024-04-25T00:00:04.127851+0000","last_became_active":"2024-04-25T00:00:02.523822+0000","last_became_peered":"2024-04-25T00:00:02.523822+0000","last_unstale":"2024-04-25T00:00:04.127851+0000","last_undegraded":"2024-04-25T00:00:04.127851+0000","last_fullsized":"2024-04-25T00:00:04.127851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:13.448915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.240497+0000","last_change":"2024-04-25T00:00:03.141296+0000","last_active":"2024-04-25T00:00:04.240497+0000","last_peered":"2024-04-25T00:00:04.240497+0000","last_clean":"2024-04-25T00:00:04.240497+0000","last_became_active":"2024-04-25T00:00:03.139916+0000","last_became_peered":"2024-04-25T00:00:03.139916+0000","last_unstale":"2024-04-25T00:00:04.240497+0000","last_undegraded":"2024-04-25T00:00:04.240497+0000","last_fullsized":"2024-04-25T00:00:04.240497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:24.534210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.670086+0000","last_change":"2024-04-25T00:00:03.149427+0000","last_active":"2024-04-25T00:00:04.670086+0000","last_peered":"2024-04-25T00:00:04.670086+0000","last_clean":"2024-04-25T00:00:04.670086+0000","last_became_active":"2024-04-25T00:00:03.148445+0000","last_became_peered":"2024-04-25T00:00:03.148445+0000","last_unstale":"2024-04-25T00:00:04.670086+0000","last_undegraded":"2024-04-25T00:00:04.670086+0000","last_fullsized":"2024-04-25T00:00:04.670086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:35.212153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128752+0000","last_change":"2024-04-25T00:00:03.146994+0000","last_active":"2024-04-25T00:00:04.128752+0000","last_peered":"2024-04-25T00:00:04.128752+0000","last_clean":"2024-04-25T00:00:04.128752+0000","last_became_active":"2024-04-25T00:00:03.146677+0000","last_became_peered":"2024-04-25T00:00:03.146677+0000","last_unstale":"2024-04-25T00:00:04.128752+0000","last_undegraded":"2024-04-25T00:00:04.128752+0000","last_fullsized":"2024-04-25T00:00:04.128752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:33.883583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.128798+0000","last_change":"2024-04-25T00:00:03.149428+0000","last_active":"2024-04-25T00:00:04.128798+0000","last_peered":"2024-04-25T00:00:04.128798+0000","last_clean":"2024-04-25T00:00:04.128798+0000","last_became_active":"2024-04-25T00:00:03.147978+0000","last_became_peered":"2024-04-25T00:00:03.147978+0000","last_unstale":"2024-04-25T00:00:04.128798+0000","last_undegraded":"2024-04-25T00:00:04.128798+0000","last_fullsized":"2024-04-25T00:00:04.128798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:34.684463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238925+0000","last_change":"2024-04-25T00:00:03.147528+0000","last_active":"2024-04-25T00:00:04.238925+0000","last_peered":"2024-04-25T00:00:04.238925+0000","last_clean":"2024-04-25T00:00:04.238925+0000","last_became_active":"2024-04-25T00:00:03.147182+0000","last_became_peered":"2024-04-25T00:00:03.147182+0000","last_unstale":"2024-04-25T00:00:04.238925+0000","last_undegraded":"2024-04-25T00:00:04.238925+0000","last_fullsized":"2024-04-25T00:00:04.238925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:27.592335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127946+0000","last_change":"2024-04-25T00:00:03.143274+0000","last_active":"2024-04-25T00:00:04.127946+0000","last_peered":"2024-04-25T00:00:04.127946+0000","last_clean":"2024-04-25T00:00:04.127946+0000","last_became_active":"2024-04-25T00:00:03.142384+0000","last_became_peered":"2024-04-25T00:00:03.142384+0000","last_unstale":"2024-04-25T00:00:04.127946+0000","last_undegraded":"2024-04-25T00:00:04.127946+0000","last_fullsized":"2024-04-25T00:00:04.127946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:41.118123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.670152+0000","last_change":"2024-04-25T00:00:03.147769+0000","last_active":"2024-04-25T00:00:04.670152+0000","last_peered":"2024-04-25T00:00:04.670152+0000","last_clean":"2024-04-25T00:00:04.670152+0000","last_became_active":"2024-04-25T00:00:03.147009+0000","last_became_peered":"2024-04-25T00:00:03.147009+0000","last_unstale":"2024-04-25T00:00:04.670152+0000","last_undegraded":"2024-04-25T00:00:04.670152+0000","last_fullsized":"2024-04-25T00:00:04.670152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:18.917532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.129268+0000","last_change":"2024-04-25T00:00:03.141519+0000","last_active":"2024-04-25T00:00:04.129268+0000","last_peered":"2024-04-25T00:00:04.129268+0000","last_clean":"2024-04-25T00:00:04.129268+0000","last_became_active":"2024-04-25T00:00:03.139391+0000","last_became_peered":"2024-04-25T00:00:03.139391+0000","last_unstale":"2024-04-25T00:00:04.129268+0000","last_undegraded":"2024-04-25T00:00:04.129268+0000","last_fullsized":"2024-04-25T00:00:04.129268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:13.043899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238914+0000","last_change":"2024-04-25T00:00:03.151311+0000","last_active":"2024-04-25T00:00:04.238914+0000","last_peered":"2024-04-25T00:00:04.238914+0000","last_clean":"2024-04-25T00:00:04.238914+0000","last_became_active":"2024-04-25T00:00:03.150992+0000","last_became_peered":"2024-04-25T00:00:03.150992+0000","last_unstale":"2024-04-25T00:00:04.238914+0000","last_undegraded":"2024-04-25T00:00:04.238914+0000","last_fullsized":"2024-04-25T00:00:04.238914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:47.572641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239200+0000","last_change":"2024-04-25T00:00:03.145623+0000","last_active":"2024-04-25T00:00:04.239200+0000","last_peered":"2024-04-25T00:00:04.239200+0000","last_clean":"2024-04-25T00:00:04.239200+0000","last_became_active":"2024-04-25T00:00:03.145260+0000","last_became_peered":"2024-04-25T00:00:03.145260+0000","last_unstale":"2024-04-25T00:00:04.239200+0000","last_undegraded":"2024-04-25T00:00:04.239200+0000","last_fullsized":"2024-04-25T00:00:04.239200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:16.985732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.238477+0000","last_change":"2024-04-25T00:00:03.144957+0000","last_active":"2024-04-25T00:00:04.238477+0000","last_peered":"2024-04-25T00:00:04.238477+0000","last_clean":"2024-04-25T00:00:04.238477+0000","last_became_active":"2024-04-25T00:00:03.144631+0000","last_became_peered":"2024-04-25T00:00:03.144631+0000","last_unstale":"2024-04-25T00:00:04.238477+0000","last_undegraded":"2024-04-25T00:00:04.238477+0000","last_fullsized":"2024-04-25T00:00:04.238477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.636553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.239170+0000","last_change":"2024-04-25T00:00:03.146574+0000","last_active":"2024-04-25T00:00:04.239170+0000","last_peered":"2024-04-25T00:00:04.239170+0000","last_clean":"2024-04-25T00:00:04.239170+0000","last_became_active":"2024-04-25T00:00:03.145095+0000","last_became_peered":"2024-04-25T00:00:03.145095+0000","last_unstale":"2024-04-25T00:00:04.239170+0000","last_undegraded":"2024-04-25T00:00:04.239170+0000","last_fullsized":"2024-04-25T00:00:04.239170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:38.060170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.127893+0000","last_change":"2024-04-25T00:00:03.143958+0000","last_active":"2024-04-25T00:00:04.127893+0000","last_peered":"2024-04-25T00:00:04.127893+0000","last_clean":"2024-04-25T00:00:04.127893+0000","last_became_active":"2024-04-25T00:00:03.143621+0000","last_became_peered":"2024-04-25T00:00:03.143621+0000","last_unstale":"2024-04-25T00:00:04.127893+0000","last_undegraded":"2024-04-25T00:00:04.127893+0000","last_fullsized":"2024-04-25T00:00:04.127893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:21.833252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T00:00:04.670043+0000","last_change":"2024-04-25T00:00:03.143703+0000","last_active":"2024-04-25T00:00:04.670043+0000","last_peered":"2024-04-25T00:00:04.670043+0000","last_clean":"2024-04-25T00:00:04.670043+0000","last_became_active":"2024-04-25T00:00:03.143400+0000","last_became_peered":"2024-04-25T00:00:03.143400+0000","last_unstale":"2024-04-25T00:00:04.670043+0000","last_undegraded":"2024-04-25T00:00:04.670043+0000","last_fullsized":"2024-04-25T00:00:04.670043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:53.984769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284249,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":116,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":15632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":1030681,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":440789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:00:10.584 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:00:10.584 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T00:00:10.585 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T00:00:10.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:10 smithi173 ceph-mon[32402]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:10.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:10 smithi173 ceph-mon[32402]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:10.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:10 smithi173 ceph-mon[32402]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:00:10.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:10 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2349687054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:00:10.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:10 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2388758852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:10.855 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:10.977 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:10 smithi110 ceph-mon[29187]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:10.978 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:10 smithi110 ceph-mon[29187]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:10.978 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:10 smithi110 ceph-mon[29187]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:00:10.978 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:10 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2349687054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:00:10.978 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:10 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2388758852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:10 smithi188 ceph-mon[33285]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:10 smithi188 ceph-mon[33285]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:00:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:10 smithi188 ceph-mon[33285]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:00:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:10 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2349687054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:00:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:10 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2388758852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:11.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:11 smithi173 ceph-mon[32402]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:00:11.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:11 smithi173 ceph-mon[32402]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-04-25T00:00:11.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:11 smithi173 ceph-mon[32402]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:00:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:11 smithi188 ceph-mon[33285]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:00:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:11 smithi188 ceph-mon[33285]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-04-25T00:00:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:11 smithi188 ceph-mon[33285]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:00:12.023 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:11 smithi110 ceph-mon[29187]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:00:12.023 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:11 smithi110 ceph-mon[29187]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.8 KiB/s wr, 15 op/s 2024-04-25T00:00:12.024 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:11 smithi110 ceph-mon[29187]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:00:12.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 -- 172.21.15.110:0/729471162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c103550 msgr2=0x7f990c1039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 --2- 172.21.15.110:0/729471162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c103550 0x7f990c1039b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9900009990 tx=0x7f9900009ca0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 -- 172.21.15.110:0/729471162 shutdown_connections 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 --2- 172.21.15.110:0/729471162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f990c103ef0 0x7f990c10a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 --2- 172.21.15.110:0/729471162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f990c100570 0x7f990c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 --2- 172.21.15.110:0/729471162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c103550 0x7f990c1039b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 -- 172.21.15.110:0/729471162 >> 172.21.15.110:0/729471162 conn(0x7f990c0fbed0 msgr2=0x7f990c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 -- 172.21.15.110:0/729471162 shutdown_connections 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 -- 172.21.15.110:0/729471162 wait complete. 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.636+0000 7f9912975700 1 Processor -- start 2024-04-25T00:00:12.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f9912975700 1 -- start start 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f9912975700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c100570 0x7f990c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f9912975700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f990c103550 0x7f990c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f9912975700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f990c103ef0 0x7f990c1a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f9912975700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1027e0 con 0x7f990c100570 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f9912975700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f990c102660 con 0x7f990c103ef0 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f9912975700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f990c102960 con 0x7f990c103550 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c100570 0x7f990c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c100570 0x7f990c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34612/0 (socket says 172.21.15.110:34612) 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 -- 172.21.15.110:0/1324930138 learned_addr learned my addr 172.21.15.110:0/1324930138 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:12.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 -- 172.21.15.110:0/1324930138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f990c103550 msgr2=0x7f990c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f990c103550 0x7f990c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 -- 172.21.15.110:0/1324930138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f990c103ef0 msgr2=0x7f990c1a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f990c103ef0 0x7f990c1a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 -- 172.21.15.110:0/1324930138 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c1a2240 con 0x7f990c100570 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.637+0000 7f990bfff700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c100570 0x7f990c19a5e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00b530 tx=0x7f98fc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.638+0000 7f99097fa700 1 -- 172.21.15.110:0/1324930138 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc015020 con 0x7f990c100570 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.638+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c1a2530 con 0x7f990c100570 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.638+0000 7f99097fa700 1 -- 172.21.15.110:0/1324930138 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98fc011ce0 con 0x7f990c100570 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.638+0000 7f99097fa700 1 -- 172.21.15.110:0/1324930138 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc010d60 con 0x7f990c100570 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.638+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c1a2b40 con 0x7f990c100570 2024-04-25T00:00:12.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.639+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990c0507a0 con 0x7f990c100570 2024-04-25T00:00:12.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.640+0000 7f99097fa700 1 -- 172.21.15.110:0/1324930138 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f98fc018020 con 0x7f990c100570 2024-04-25T00:00:12.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.640+0000 7f99097fa700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f98f40a2d60 0x7f98f40a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:12.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.640+0000 7f990b7fe700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f98f40a2d60 0x7f98f40a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:12.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.640+0000 7f99097fa700 1 -- 172.21.15.110:0/1324930138 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f98fc0cec80 con 0x7f990c100570 2024-04-25T00:00:12.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.641+0000 7f990b7fe700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f98f40a2d60 0x7f98f40a5220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9900005b40 tx=0x7f9900005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:12.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.644+0000 7f99097fa700 1 -- 172.21.15.110:0/1324930138 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f98fc01f020 con 0x7f990c100570 2024-04-25T00:00:12.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.920+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f990c02dc00 con 0x7f990c100570 2024-04-25T00:00:12.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.922+0000 7f99097fa700 1 -- 172.21.15.110:0/1324930138 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f98fc097030 con 0x7f990c100570 2024-04-25T00:00:12.922 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-04-25T00:00:12.922 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAMnSlmDqLrNhAAK+dafmpkQ+CREDa/jRrTEA== 2024-04-25T00:00:12.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f98f40a2d60 msgr2=0x7f98f40a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:12.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f98f40a2d60 0x7f98f40a5220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9900005b40 tx=0x7f9900005a90 comp rx=0 tx=0).stop 2024-04-25T00:00:12.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c100570 msgr2=0x7f990c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:12.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c100570 0x7f990c19a5e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00b530 tx=0x7f98fc00b8f0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 shutdown_connections 2024-04-25T00:00:12.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f98f40a2d60 0x7f98f40a5220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f990c103ef0 0x7f990c1a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f990c100570 0x7f990c19a5e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 --2- 172.21.15.110:0/1324930138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f990c103550 0x7f990c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:12.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 >> 172.21.15.110:0/1324930138 conn(0x7f990c0fbed0 msgr2=0x7f990c0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:12.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 shutdown_connections 2024-04-25T00:00:12.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:12.924+0000 7f9912975700 1 -- 172.21.15.110:0/1324930138 wait complete. 2024-04-25T00:00:13.631 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T00:00:13.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T00:00:13.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:00:13.667 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:00:13.667 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:00:13.668 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T00:00:13.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:13 smithi173 ceph-mon[32402]: pgmap v294: 97 pgs: 97 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2024-04-25T00:00:13.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:13 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1324930138' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:00:13.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:13 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1324930138' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:00:13.945 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:13 smithi188 ceph-mon[33285]: pgmap v294: 97 pgs: 97 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2024-04-25T00:00:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:13 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1324930138' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:00:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:13 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1324930138' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:00:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:13 smithi110 ceph-mon[29187]: pgmap v294: 97 pgs: 97 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2024-04-25T00:00:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:13 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1324930138' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:00:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:13 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1324930138' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:00:15.694 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:15 smithi110 ceph-mon[29187]: pgmap v295: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.8 KiB/s wr, 17 op/s 2024-04-25T00:00:15.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 -- 172.21.15.110:0/1540061248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c106450 msgr2=0x7f9f3c10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:15.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 --2- 172.21.15.110:0/1540061248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c106450 0x7f9f3c10e020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400b0d0 tx=0x7f9f3400b3e0 comp rx=0 tx=0).stop 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 -- 172.21.15.110:0/1540061248 shutdown_connections 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 --2- 172.21.15.110:0/1540061248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c106450 0x7f9f3c10e020 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 --2- 172.21.15.110:0/1540061248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c1013c0 0x7f9f3c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 --2- 172.21.15.110:0/1540061248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c101d70 0x7f9f3c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 -- 172.21.15.110:0/1540061248 >> 172.21.15.110:0/1540061248 conn(0x7f9f3c0780e0 msgr2=0x7f9f3c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 -- 172.21.15.110:0/1540061248 shutdown_connections 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.795+0000 7f9f41316700 1 -- 172.21.15.110:0/1540061248 wait complete. 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f41316700 1 Processor -- start 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f41316700 1 -- start start 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f41316700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c1013c0 0x7f9f3c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f41316700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c101d70 0x7f9f3c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f41316700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c106450 0x7f9f3c1a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:15.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f41316700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c111360 con 0x7f9f3c101d70 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f41316700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1111e0 con 0x7f9f3c106450 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f41316700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1114e0 con 0x7f9f3c1013c0 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c101d70 0x7f9f3c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c101d70 0x7f9f3c199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34630/0 (socket says 172.21.15.110:34630) 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3a7fc700 1 -- 172.21.15.110:0/4115975532 learned_addr learned my addr 172.21.15.110:0/4115975532 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3affd700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c1013c0 0x7f9f3c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3a7fc700 1 -- 172.21.15.110:0/4115975532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c1013c0 msgr2=0x7f9f3c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3a7fc700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c1013c0 0x7f9f3c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3a7fc700 1 -- 172.21.15.110:0/4115975532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c106450 msgr2=0x7f9f3c1a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3a7fc700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c106450 0x7f9f3c1a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.796+0000 7f9f3a7fc700 1 -- 172.21.15.110:0/4115975532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c1a4d40 con 0x7f9f3c101d70 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.797+0000 7f9f3a7fc700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c101d70 0x7f9f3c199cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000b500 tx=0x7f9f3000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:15.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.797+0000 7f9f23fff700 1 -- 172.21.15.110:0/4115975532 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30015020 con 0x7f9f3c101d70 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.797+0000 7f9f23fff700 1 -- 172.21.15.110:0/4115975532 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f30011ce0 con 0x7f9f3c101d70 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.797+0000 7f9f23fff700 1 -- 172.21.15.110:0/4115975532 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30010d40 con 0x7f9f3c101d70 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.797+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c1a4fd0 con 0x7f9f3c101d70 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.797+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c1a5430 con 0x7f9f3c101d70 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.798+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c068f20 con 0x7f9f3c101d70 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.799+0000 7f9f23fff700 1 -- 172.21.15.110:0/4115975532 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9f30018020 con 0x7f9f3c101d70 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.800+0000 7f9f23fff700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9f240a2cb0 0x7f9f240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.800+0000 7f9f3affd700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9f240a2cb0 0x7f9f240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.800+0000 7f9f23fff700 1 -- 172.21.15.110:0/4115975532 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9f300ce770 con 0x7f9f3c101d70 2024-04-25T00:00:15.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.800+0000 7f9f3affd700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9f240a2cb0 0x7f9f240a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005fd0 tx=0x7f9f2c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:15.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:15.804+0000 7f9f23fff700 1 -- 172.21.15.110:0/4115975532 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f3001f020 con 0x7f9f3c101d70 2024-04-25T00:00:15.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:15 smithi173 ceph-mon[32402]: pgmap v295: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.8 KiB/s wr, 17 op/s 2024-04-25T00:00:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:15 smithi188 ceph-mon[33285]: pgmap v295: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.8 KiB/s wr, 17 op/s 2024-04-25T00:00:16.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.058+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9f3c02dc50 con 0x7f9f3c101d70 2024-04-25T00:00:16.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.064+0000 7f9f23fff700 1 -- 172.21.15.110:0/4115975532 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263206 (secure 0 0 0) 0x7f9f3009b830 con 0x7f9f3c101d70 2024-04-25T00:00:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:16.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9f240a2cb0 msgr2=0x7f9f240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:16.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9f240a2cb0 0x7f9f240a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005fd0 tx=0x7f9f2c00a560 comp rx=0 tx=0).stop 2024-04-25T00:00:16.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c101d70 msgr2=0x7f9f3c199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:16.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c101d70 0x7f9f3c199cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000b500 tx=0x7f9f3000b8c0 comp rx=0 tx=0).stop 2024-04-25T00:00:16.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 shutdown_connections 2024-04-25T00:00:16.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9f240a2cb0 0x7f9f240a5170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:16.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c106450 0x7f9f3c1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:16.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c1013c0 0x7f9f3c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:16.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 --2- 172.21.15.110:0/4115975532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f3c101d70 0x7f9f3c199cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:16.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 >> 172.21.15.110:0/4115975532 conn(0x7f9f3c0780e0 msgr2=0x7f9f3c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:16.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.066+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 shutdown_connections 2024-04-25T00:00:16.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:16.067+0000 7f9f41316700 1 -- 172.21.15.110:0/4115975532 wait complete. 2024-04-25T00:00:16.782 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:16 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4115975532' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:00:16.812 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":546485163},{"type":"v1","addr":"172.21.15.110:6801","nonce":546485163}]},"active_addr":"172.21.15.110:6801/546485163","active_change":"2024-04-24T23:52:54.426213+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.110:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1380653271}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":2891513793}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":4162407706}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":135252951}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":2619953966}]}]} 2024-04-25T00:00:16.818 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T00:00:16.818 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T00:00:16.819 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:00:16.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:16 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4115975532' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:00:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:16 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4115975532' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:00:17.059 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:17 smithi173 ceph-mon[32402]: pgmap v296: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-25T00:00:17.923 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:17 smithi110 ceph-mon[29187]: pgmap v296: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-25T00:00:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:17 smithi188 ceph-mon[33285]: pgmap v296: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-25T00:00:18.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2095734489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1012b0 msgr2=0x7fc4ac10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2095734489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1012b0 0x7fc4ac10d770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400b0a0 tx=0x7fc4a400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2095734489 shutdown_connections 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2095734489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1012b0 0x7fc4ac10d770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2095734489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4ac1068a0 0x7fc4ac106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2095734489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4ac1008f0 0x7fc4ac100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2095734489 >> 172.21.15.110:0/2095734489 conn(0x7fc4ac0777a0 msgr2=0x7fc4ac0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2095734489 shutdown_connections 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.968+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2095734489 wait complete. 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b3ffa700 1 Processor -- start 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b3ffa700 1 -- start start 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b3ffa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4ac1008f0 0x7fc4ac199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:18.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b3ffa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4ac1012b0 0x7fc4ac199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b3ffa700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1068a0 0x7fc4ac1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b3ffa700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1112e0 con 0x7fc4ac1068a0 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b3ffa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac111160 con 0x7fc4ac1012b0 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b3ffa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac111460 con 0x7fc4ac1008f0 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.969+0000 7fc4b2597700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1068a0 0x7fc4ac1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b2597700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1068a0 0x7fc4ac1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35940/0 (socket says 172.21.15.110:35940) 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b2597700 1 -- 172.21.15.110:0/2759253796 learned_addr learned my addr 172.21.15.110:0/2759253796 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b1595700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4ac1012b0 0x7fc4ac199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b1d96700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4ac1008f0 0x7fc4ac199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b2597700 1 -- 172.21.15.110:0/2759253796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4ac1008f0 msgr2=0x7fc4ac199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b2597700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4ac1008f0 0x7fc4ac199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b2597700 1 -- 172.21.15.110:0/2759253796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4ac1012b0 msgr2=0x7fc4ac199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b2597700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4ac1012b0 0x7fc4ac199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:18.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b2597700 1 -- 172.21.15.110:0/2759253796 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac1a4c00 con 0x7fc4ac1068a0 2024-04-25T00:00:18.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b1d96700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4ac1008f0 0x7fc4ac199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:18.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b2597700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1068a0 0x7fc4ac1a46c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40062a0 tx=0x7fc4a4004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:18.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4a2ffd700 1 -- 172.21.15.110:0/2759253796 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4021070 con 0x7fc4ac1068a0 2024-04-25T00:00:18.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4a2ffd700 1 -- 172.21.15.110:0/2759253796 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4a4007c80 con 0x7fc4ac1068a0 2024-04-25T00:00:18.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4a2ffd700 1 -- 172.21.15.110:0/2759253796 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a401d450 con 0x7fc4ac1068a0 2024-04-25T00:00:18.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ac1a4dd0 con 0x7fc4ac1068a0 2024-04-25T00:00:18.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.970+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ac1a52d0 con 0x7fc4ac1068a0 2024-04-25T00:00:18.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.971+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ac101fe0 con 0x7fc4ac1068a0 2024-04-25T00:00:18.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.972+0000 7fc4a2ffd700 1 -- 172.21.15.110:0/2759253796 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc4a4003a00 con 0x7fc4ac1068a0 2024-04-25T00:00:18.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.973+0000 7fc4a2ffd700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4980a2d30 0x7fc4980a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:18.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.973+0000 7fc4b1d96700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4980a2d30 0x7fc4980a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:18.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.973+0000 7fc4a2ffd700 1 -- 172.21.15.110:0/2759253796 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc4a400f020 con 0x7fc4ac1068a0 2024-04-25T00:00:18.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.974+0000 7fc4b1d96700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4980a2d30 0x7fc4980a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8005fd0 tx=0x7fc4a8005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:18.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:18.977+0000 7fc4a2ffd700 1 -- 172.21.15.110:0/2759253796 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc4a402a020 con 0x7fc4ac1068a0 2024-04-25T00:00:19.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.203+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4ac02db30 con 0x7fc4ac1068a0 2024-04-25T00:00:19.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.204+0000 7fc4a2ffd700 1 -- 172.21.15.110:0/2759253796 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22267 (secure 0 0 0) 0x7fc4a409d710 con 0x7fc4ac1068a0 2024-04-25T00:00:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:19.205 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":85,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:04.124840+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.206+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4980a2d30 msgr2=0x7fc4980a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.206+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4980a2d30 0x7fc4980a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8005fd0 tx=0x7fc4a8005db0 comp rx=0 tx=0).stop 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.206+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1068a0 msgr2=0x7fc4ac1a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.206+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1068a0 0x7fc4ac1a46c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40062a0 tx=0x7fc4a4004e20 comp rx=0 tx=0).stop 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.207+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 shutdown_connections 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.207+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4980a2d30 0x7fc4980a51f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.207+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4ac1068a0 0x7fc4ac1a46c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.207+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4ac1008f0 0x7fc4ac199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.207+0000 7fc4b3ffa700 1 --2- 172.21.15.110:0/2759253796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4ac1012b0 0x7fc4ac199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.207+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 >> 172.21.15.110:0/2759253796 conn(0x7fc4ac0777a0 msgr2=0x7fc4ac103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:19.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.207+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 shutdown_connections 2024-04-25T00:00:19.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:19.207+0000 7fc4b3ffa700 1 -- 172.21.15.110:0/2759253796 wait complete. 2024-04-25T00:00:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:19 smithi173 ceph-mon[32402]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-04-25T00:00:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:19 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2759253796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:19.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:19 smithi110 ceph-mon[29187]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-04-25T00:00:19.918 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:19 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2759253796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:19.945 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T00:00:19.946 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:00:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:19 smithi188 ceph-mon[33285]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-04-25T00:00:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:19 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2759253796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:20.185 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:21.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:21 smithi110 ceph-mon[29187]: pgmap v298: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-25T00:00:21.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:21 smithi173 ceph-mon[32402]: pgmap v298: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-25T00:00:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:21 smithi188 ceph-mon[33285]: pgmap v298: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-25T00:00:22.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 -- 172.21.15.110:0/2158616364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c106480 msgr2=0x7fbd1c10e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 --2- 172.21.15.110:0/2158616364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c106480 0x7fbd1c10e050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400b0d0 tx=0x7fbd1400b3e0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 -- 172.21.15.110:0/2158616364 shutdown_connections 2024-04-25T00:00:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 --2- 172.21.15.110:0/2158616364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c106480 0x7fbd1c10e050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 --2- 172.21.15.110:0/2158616364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1013c0 0x7fbd1c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 --2- 172.21.15.110:0/2158616364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd1c101d70 0x7fbd1c105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 -- 172.21.15.110:0/2158616364 >> 172.21.15.110:0/2158616364 conn(0x7fbd1c0771d0 msgr2=0x7fbd1c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 -- 172.21.15.110:0/2158616364 shutdown_connections 2024-04-25T00:00:22.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.067+0000 7fbd229a8700 1 -- 172.21.15.110:0/2158616364 wait complete. 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd229a8700 1 Processor -- start 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd229a8700 1 -- start start 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd229a8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1013c0 0x7fbd1c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd229a8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c101d70 0x7fbd1c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd229a8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd1c106480 0x7fbd1c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd229a8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c111390 con 0x7fbd1c101d70 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd229a8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c111210 con 0x7fbd1c106480 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd229a8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c111510 con 0x7fbd1c1013c0 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.068+0000 7fbd1b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c101d70 0x7fbd1c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c101d70 0x7fbd1c19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35946/0 (socket says 172.21.15.110:35946) 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1b7fe700 1 -- 172.21.15.110:0/1512860319 learned_addr learned my addr 172.21.15.110:0/1512860319 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1bfff700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1013c0 0x7fbd1c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd20f45700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd1c106480 0x7fbd1c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:22.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1b7fe700 1 -- 172.21.15.110:0/1512860319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1013c0 msgr2=0x7fbd1c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1b7fe700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1013c0 0x7fbd1c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1b7fe700 1 -- 172.21.15.110:0/1512860319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd1c106480 msgr2=0x7fbd1c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1b7fe700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd1c106480 0x7fbd1c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1b7fe700 1 -- 172.21.15.110:0/1512860319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c199030 con 0x7fbd1c101d70 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd1b7fe700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c101d70 0x7fbd1c19ebc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000c8f0 tx=0x7fbd1000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd197fa700 1 -- 172.21.15.110:0/1512860319 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10007940 con 0x7fbd1c101d70 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd197fa700 1 -- 172.21.15.110:0/1512860319 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd1000f450 con 0x7fbd1c101d70 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c199320 con 0x7fbd1c101d70 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd197fa700 1 -- 172.21.15.110:0/1512860319 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10018660 con 0x7fbd1c101d70 2024-04-25T00:00:22.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.069+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1a5600 con 0x7fbd1c101d70 2024-04-25T00:00:22.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.070+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1c102880 con 0x7fbd1c101d70 2024-04-25T00:00:22.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.071+0000 7fbd197fa700 1 -- 172.21.15.110:0/1512860319 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbd10007ae0 con 0x7fbd1c101d70 2024-04-25T00:00:22.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.072+0000 7fbd197fa700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbd040a2d30 0x7fbd040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:22.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.072+0000 7fbd1bfff700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbd040a2d30 0x7fbd040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:22.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.072+0000 7fbd197fa700 1 -- 172.21.15.110:0/1512860319 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbd100cee50 con 0x7fbd1c101d70 2024-04-25T00:00:22.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.072+0000 7fbd1bfff700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbd040a2d30 0x7fbd040a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009a10 tx=0x7fbd0c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:22.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.076+0000 7fbd197fa700 1 -- 172.21.15.110:0/1512860319 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbd10014030 con 0x7fbd1c101d70 2024-04-25T00:00:22.275 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.274+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd1c199c80 con 0x7fbd1c101d70 2024-04-25T00:00:22.276 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.276+0000 7fbd197fa700 1 -- 172.21.15.110:0/1512860319 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22267 (secure 0 0 0) 0x7fbd1009be00 con 0x7fbd1c101d70 2024-04-25T00:00:22.277 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:00:22.277 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":85,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:04.124840+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:00:22.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbd040a2d30 msgr2=0x7fbd040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:22.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbd040a2d30 0x7fbd040a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009a10 tx=0x7fbd0c009190 comp rx=0 tx=0).stop 2024-04-25T00:00:22.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c101d70 msgr2=0x7fbd1c19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:22.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c101d70 0x7fbd1c19ebc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000c8f0 tx=0x7fbd1000cc00 comp rx=0 tx=0).stop 2024-04-25T00:00:22.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 shutdown_connections 2024-04-25T00:00:22.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbd040a2d30 0x7fbd040a51f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd1c106480 0x7fbd1c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c1013c0 0x7fbd1c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 --2- 172.21.15.110:0/1512860319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd1c101d70 0x7fbd1c19ebc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:22.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 >> 172.21.15.110:0/1512860319 conn(0x7fbd1c0771d0 msgr2=0x7fbd1c06abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:22.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 shutdown_connections 2024-04-25T00:00:22.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:22.279+0000 7fbd229a8700 1 -- 172.21.15.110:0/1512860319 wait complete. 2024-04-25T00:00:22.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:22 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1512860319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:22.998 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:22 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1512860319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:22 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1512860319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:00:23.027 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T00:00:23.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T00:00:23.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T00:00:23.029 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T00:00:23.029 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T00:00:23.029 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T00:00:23.030 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T00:00:23.030 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T00:00:23.031 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T00:00:23.031 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T00:00:23.031 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T00:00:23.032 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T00:00:23.610 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.706 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.708 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.716 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.726 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.727 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.731 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.731 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.732 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:23.850 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:23 smithi110 ceph-mon[29187]: pgmap v299: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-25T00:00:23.851 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:23 smithi173 ceph-mon[32402]: pgmap v299: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-25T00:00:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:23 smithi188 ceph-mon[33285]: pgmap v299: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-25T00:00:25.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:25 smithi110 ceph-mon[29187]: pgmap v300: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-04-25T00:00:25.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:25 smithi173 ceph-mon[32402]: pgmap v300: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-04-25T00:00:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:25 smithi188 ceph-mon[33285]: pgmap v300: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-04-25T00:00:26.641 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:27.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:27 smithi110 ceph-mon[29187]: pgmap v301: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T00:00:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:27 smithi188 ceph-mon[33285]: pgmap v301: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T00:00:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:27 smithi173 ceph-mon[32402]: pgmap v301: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T00:00:29.603 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:29.705 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:29 smithi110 ceph-mon[29187]: pgmap v302: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T00:00:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:29 smithi188 ceph-mon[33285]: pgmap v302: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T00:00:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:29 smithi173 ceph-mon[32402]: pgmap v302: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T00:00:30.240 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:31.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 -- 172.21.15.110:0/1663684750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4107540 msgr2=0x7f24a410f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:31.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 --2- 172.21.15.110:0/1663684750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4107540 0x7f24a410f110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f249c00b0d0 tx=0x7f249c00b3e0 comp rx=0 tx=0).stop 2024-04-25T00:00:31.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 -- 172.21.15.110:0/1663684750 shutdown_connections 2024-04-25T00:00:31.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 --2- 172.21.15.110:0/1663684750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4107540 0x7f24a410f110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 --2- 172.21.15.110:0/1663684750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a4102550 0x7f24a4102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 --2- 172.21.15.110:0/1663684750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24a4102f00 0x7f24a4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 -- 172.21.15.110:0/1663684750 >> 172.21.15.110:0/1663684750 conn(0x7f24a40793f0 msgr2=0x7f24a4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 -- 172.21.15.110:0/1663684750 shutdown_connections 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.505+0000 7f24aa009700 1 -- 172.21.15.110:0/1663684750 wait complete. 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24aa009700 1 Processor -- start 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24aa009700 1 -- start start 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24aa009700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4102550 0x7f24a4074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24aa009700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a4102f00 0x7f24a4074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24aa009700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24a4107540 0x7f24a4074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24aa009700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4112450 con 0x7f24a4102550 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24aa009700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41122d0 con 0x7f24a4102f00 2024-04-25T00:00:31.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24aa009700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41125d0 con 0x7f24a4107540 2024-04-25T00:00:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.506+0000 7f24a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4102550 0x7f24a4074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4102550 0x7f24a4074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56214/0 (socket says 172.21.15.110:56214) 2024-04-25T00:00:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a37fe700 1 -- 172.21.15.110:0/2557109367 learned_addr learned my addr 172.21.15.110:0/2557109367 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:31.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a2ffd700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a4102f00 0x7f24a4074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a3fff700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24a4107540 0x7f24a4074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a37fe700 1 -- 172.21.15.110:0/2557109367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24a4107540 msgr2=0x7f24a4074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a37fe700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24a4107540 0x7f24a4074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a37fe700 1 -- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a4102f00 msgr2=0x7f24a4074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a37fe700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a4102f00 0x7f24a4074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a37fe700 1 -- 172.21.15.110:0/2557109367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a41a6e10 con 0x7f24a4102550 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a2ffd700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a4102f00 0x7f24a4074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a37fe700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4102550 0x7f24a4074530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f249400c9b0 tx=0x7f249400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494007b00 con 0x7f24a4102550 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a41a7100 con 0x7f24a4102550 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2494007ca0 con 0x7f24a4102550 2024-04-25T00:00:31.577 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494018730 con 0x7f24a4102550 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.507+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41a7690 con 0x7f24a4102550 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.508+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24a4062940 con 0x7f24a4102550 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.509+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f249400f450 con 0x7f24a4102550 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.510+0000 7f24a0ff9700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f248c0a2cb0 0x7f248c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.510+0000 7f24a2ffd700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f248c0a2cb0 0x7f248c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.510+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f24940cf4f0 con 0x7f24a4102550 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.510+0000 7f24a0ff9700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] conn(0x7f248c0ac490 0x7f248c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.510+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 --> [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f248c0aef60 con 0x7f248c0ac490 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.510+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f24940cf9d0 con 0x7f24a4102550 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.510+0000 7f24a2ffd700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f248c0a2cb0 0x7f248c0a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f24a4108980 tx=0x7f2498008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.510+0000 7f24a3fff700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] conn(0x7f248c0ac490 0x7f248c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.511+0000 7f24a3fff700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] conn(0x7f248c0ac490 0x7f248c0ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:31.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.512+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 <== osd.1 v2:172.21.15.173:6800/1633800415 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f248c0aef60 con 0x7f248c0ac490 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.528+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 --> [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24a4062d40 con 0x7f248c0ac490 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.528+0000 7f24a0ff9700 1 -- 172.21.15.110:0/2557109367 <== osd.1 v2:172.21.15.173:6800/1633800415 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f24a4062d40 con 0x7f248c0ac490 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] conn(0x7f248c0ac490 msgr2=0x7f248c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] conn(0x7f248c0ac490 0x7f248c0ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f248c0a2cb0 msgr2=0x7f248c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f248c0a2cb0 0x7f248c0a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f24a4108980 tx=0x7f2498008040 comp rx=0 tx=0).stop 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4102550 msgr2=0x7f24a4074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 --2- 172.21.15.110:0/2557109367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24a4102550 0x7f24a4074530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f249400c9b0 tx=0x7f249400cd70 comp rx=0 tx=0).stop 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24a3fff700 1 -- 172.21.15.110:0/2557109367 reap_dead start 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 shutdown_connections 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 >> 172.21.15.110:0/2557109367 conn(0x7f24a40793f0 msgr2=0x7f24a41002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 shutdown_connections 2024-04-25T00:00:31.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:31.529+0000 7f24aa009700 1 -- 172.21.15.110:0/2557109367 wait complete. 2024-04-25T00:00:31.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:31 smithi110 ceph-mon[29187]: pgmap v303: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T00:00:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:31 smithi188 ceph-mon[33285]: pgmap v303: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T00:00:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:31 smithi173 ceph-mon[32402]: pgmap v303: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T00:00:33.850 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:33 smithi110 ceph-mon[29187]: pgmap v304: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T00:00:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:33 smithi188 ceph-mon[33285]: pgmap v304: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T00:00:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:33 smithi173 ceph-mon[32402]: pgmap v304: 97 pgs: 97 active+clean; 633 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T00:00:34.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 -- 172.21.15.110:0/3749741370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 msgr2=0x7faf241077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 --2- 172.21.15.110:0/3749741370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 0x7faf241077e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faf180099c0 tx=0x7faf18009cd0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 -- 172.21.15.110:0/3749741370 shutdown_connections 2024-04-25T00:00:34.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 --2- 172.21.15.110:0/3749741370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf241026a0 0x7faf2410e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 --2- 172.21.15.110:0/3749741370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 0x7faf241077e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 --2- 172.21.15.110:0/3749741370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf24101ce0 0x7faf24102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 -- 172.21.15.110:0/3749741370 >> 172.21.15.110:0/3749741370 conn(0x7faf24076450 msgr2=0x7faf24076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 -- 172.21.15.110:0/3749741370 shutdown_connections 2024-04-25T00:00:34.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.166+0000 7faf2a8a4700 1 -- 172.21.15.110:0/3749741370 wait complete. 2024-04-25T00:00:34.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.167+0000 7faf2a8a4700 1 Processor -- start 2024-04-25T00:00:34.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.167+0000 7faf2a8a4700 1 -- start start 2024-04-25T00:00:34.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.167+0000 7faf2a8a4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf24101ce0 0x7faf24199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.167+0000 7faf2a8a4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf241026a0 0x7faf24199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.167+0000 7faf2a8a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 0x7faf241a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.167+0000 7faf2a8a4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf24111480 con 0x7faf24107400 2024-04-25T00:00:34.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.167+0000 7faf2a8a4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf24111300 con 0x7faf241026a0 2024-04-25T00:00:34.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.167+0000 7faf2a8a4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf24111600 con 0x7faf24101ce0 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 0x7faf241a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 0x7faf241a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56238/0 (socket says 172.21.15.110:56238) 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 -- 172.21.15.110:0/4256425813 learned_addr learned my addr 172.21.15.110:0/4256425813 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf237fe700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf241026a0 0x7faf24199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf23fff700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf24101ce0 0x7faf24199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 -- 172.21.15.110:0/4256425813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf24101ce0 msgr2=0x7faf24199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf24101ce0 0x7faf24199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 -- 172.21.15.110:0/4256425813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf241026a0 msgr2=0x7faf24199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf241026a0 0x7faf24199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 -- 172.21.15.110:0/4256425813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf241a4de0 con 0x7faf24107400 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf237fe700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf241026a0 0x7faf24199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf28e41700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 0x7faf241a48a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faf1c007d60 tx=0x7faf1c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00ad40 con 0x7faf24107400 2024-04-25T00:00:34.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf1c004030 con 0x7faf24107400 2024-04-25T00:00:34.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf241a5010 con 0x7faf24107400 2024-04-25T00:00:34.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c012680 con 0x7faf24107400 2024-04-25T00:00:34.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.168+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf241a5520 con 0x7faf24107400 2024-04-25T00:00:34.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.169+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf240ff580 con 0x7faf24107400 2024-04-25T00:00:34.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.170+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7faf1c003a00 con 0x7faf24107400 2024-04-25T00:00:34.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.171+0000 7faf217fa700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faf0c0a2cb0 0x7faf0c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.171+0000 7faf23fff700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faf0c0a2cb0 0x7faf0c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.171+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7faf1c0cf9f0 con 0x7faf24107400 2024-04-25T00:00:34.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.171+0000 7faf217fa700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] conn(0x7faf0c0ac490 0x7faf0c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:34.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.171+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 --> [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf0c0aef60 con 0x7faf0c0ac490 2024-04-25T00:00:34.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.171+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7faf1c0cfed0 con 0x7faf24107400 2024-04-25T00:00:34.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.171+0000 7faf237fe700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] conn(0x7faf0c0ac490 0x7faf0c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:34.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.171+0000 7faf23fff700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faf0c0a2cb0 0x7faf0c0a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf18009750 tx=0x7faf18005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.172+0000 7faf237fe700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] conn(0x7faf0c0ac490 0x7faf0c0ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:34.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.172+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 <== osd.0 v2:172.21.15.110:6802/1941809226 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf0c0aef60 con 0x7faf0c0ac490 2024-04-25T00:00:34.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.189+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 --> [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf2406a580 con 0x7faf0c0ac490 2024-04-25T00:00:34.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.189+0000 7faf217fa700 1 -- 172.21.15.110:0/4256425813 <== osd.0 v2:172.21.15.110:6802/1941809226 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf2406a580 con 0x7faf0c0ac490 2024-04-25T00:00:34.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.189+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] conn(0x7faf0c0ac490 msgr2=0x7faf0c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.189+0000 7faf2a8a4700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] conn(0x7faf0c0ac490 0x7faf0c0ae8b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.189+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faf0c0a2cb0 msgr2=0x7faf0c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.189+0000 7faf2a8a4700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faf0c0a2cb0 0x7faf0c0a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf18009750 tx=0x7faf18005fd0 comp rx=0 tx=0).stop 2024-04-25T00:00:34.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.189+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 msgr2=0x7faf241a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:34.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.189+0000 7faf2a8a4700 1 --2- 172.21.15.110:0/4256425813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf24107400 0x7faf241a48a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faf1c007d60 tx=0x7faf1c00a340 comp rx=0 tx=0).stop 2024-04-25T00:00:34.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.190+0000 7faf28e41700 1 -- 172.21.15.110:0/4256425813 reap_dead start 2024-04-25T00:00:34.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.190+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 shutdown_connections 2024-04-25T00:00:34.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.190+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 >> 172.21.15.110:0/4256425813 conn(0x7faf24076450 msgr2=0x7faf240febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:34.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.190+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 shutdown_connections 2024-04-25T00:00:34.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:34.190+0000 7faf2a8a4700 1 -- 172.21.15.110:0/4256425813 wait complete. 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 -- 172.21.15.110:0/2202196734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481064c0 msgr2=0x7fa74810e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 --2- 172.21.15.110:0/2202196734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481064c0 0x7fa74810e090 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa74000b0d0 tx=0x7fa74000b3e0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 -- 172.21.15.110:0/2202196734 shutdown_connections 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 --2- 172.21.15.110:0/2202196734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481064c0 0x7fa74810e090 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 --2- 172.21.15.110:0/2202196734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7481014c0 0x7fa7481018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 --2- 172.21.15.110:0/2202196734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa748101e70 0x7fa748105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 -- 172.21.15.110:0/2202196734 >> 172.21.15.110:0/2202196734 conn(0x7fa748077040 msgr2=0x7fa748077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 -- 172.21.15.110:0/2202196734 shutdown_connections 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.526+0000 7fa74d807700 1 -- 172.21.15.110:0/2202196734 wait complete. 2024-04-25T00:00:35.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa74d807700 1 Processor -- start 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa74d807700 1 -- start start 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa74d807700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481014c0 0x7fa748199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa74d807700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa748101e70 0x7fa748199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa74d807700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7481064c0 0x7fa7481a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa74d807700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7481113d0 con 0x7fa7481014c0 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa74d807700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa748111250 con 0x7fa7481064c0 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa74d807700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa748111550 con 0x7fa748101e70 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.527+0000 7fa746ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481014c0 0x7fa748199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa746ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481014c0 0x7fa748199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56254/0 (socket says 172.21.15.110:56254) 2024-04-25T00:00:35.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa746ffd700 1 -- 172.21.15.110:0/1018241286 learned_addr learned my addr 172.21.15.110:0/1018241286 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa7477fe700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7481064c0 0x7fa7481a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa7467fc700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa748101e70 0x7fa748199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa746ffd700 1 -- 172.21.15.110:0/1018241286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa748101e70 msgr2=0x7fa748199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa746ffd700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa748101e70 0x7fa748199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa746ffd700 1 -- 172.21.15.110:0/1018241286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7481064c0 msgr2=0x7fa7481a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa746ffd700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7481064c0 0x7fa7481a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa746ffd700 1 -- 172.21.15.110:0/1018241286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7481a4e10 con 0x7fa7481014c0 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa746ffd700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481014c0 0x7fa748199850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b530 tx=0x7fa73c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c015020 con 0x7fa7481014c0 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7481a4fe0 con 0x7fa7481014c0 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa73c011ce0 con 0x7fa7481014c0 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c010d60 con 0x7fa7481014c0 2024-04-25T00:00:35.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.528+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7481a5490 con 0x7fa7481014c0 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.530+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa748062940 con 0x7fa7481014c0 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.530+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa73c018020 con 0x7fa7481014c0 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.531+0000 7fa74c805700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7300a2cb0 0x7fa7300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.531+0000 7fa7467fc700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7300a2cb0 0x7fa7300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.531+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa73c0ce620 con 0x7fa7481014c0 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.532+0000 7fa74c805700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] conn(0x7fa7300ac490 0x7fa7300ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.532+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 --> [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7300aef60 con 0x7fa7300ac490 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.532+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fa73c0d5050 con 0x7fa7481014c0 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.532+0000 7fa7467fc700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7300a2cb0 0x7fa7300a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa738005fd0 tx=0x7fa73800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.532+0000 7fa7477fe700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] conn(0x7fa7300ac490 0x7fa7300ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.532+0000 7fa7477fe700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] conn(0x7fa7300ac490 0x7fa7300ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.533+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 <== osd.4 v2:172.21.15.173:6808/3034286284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7300aef60 con 0x7fa7300ac490 2024-04-25T00:00:35.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 --> [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa748062d40 con 0x7fa7300ac490 2024-04-25T00:00:35.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74c805700 1 -- 172.21.15.110:0/1018241286 <== osd.4 v2:172.21.15.173:6808/3034286284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa748062d40 con 0x7fa7300ac490 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 >> [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] conn(0x7fa7300ac490 msgr2=0x7fa7300ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] conn(0x7fa7300ac490 0x7fa7300ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7300a2cb0 msgr2=0x7fa7300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7300a2cb0 0x7fa7300a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa738005fd0 tx=0x7fa73800a560 comp rx=0 tx=0).stop 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481014c0 msgr2=0x7fa748199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 --2- 172.21.15.110:0/1018241286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7481014c0 0x7fa748199850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b530 tx=0x7fa73c00b8f0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa7477fe700 1 -- 172.21.15.110:0/1018241286 reap_dead start 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 shutdown_connections 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.550+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 >> 172.21.15.110:0/1018241286 conn(0x7fa748077040 msgr2=0x7fa7480ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.551+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 shutdown_connections 2024-04-25T00:00:35.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.551+0000 7fa74d807700 1 -- 172.21.15.110:0/1018241286 wait complete. 2024-04-25T00:00:35.735 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:35 smithi110 ceph-mon[29187]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T00:00:35.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 -- 172.21.15.110:0/1317318197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb160101b20 msgr2=0x7fb16010dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 --2- 172.21.15.110:0/1317318197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb160101b20 0x7fb16010dfe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b0a0 tx=0x7fb15800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 -- 172.21.15.110:0/1317318197 shutdown_connections 2024-04-25T00:00:35.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 --2- 172.21.15.110:0/1317318197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb160101b20 0x7fb16010dfe0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 --2- 172.21.15.110:0/1317318197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1601071e0 0x7fb1601075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 --2- 172.21.15.110:0/1317318197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb160101160 0x7fb1601015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 -- 172.21.15.110:0/1317318197 >> 172.21.15.110:0/1317318197 conn(0x7fb1600771a0 msgr2=0x7fb1600775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 -- 172.21.15.110:0/1317318197 shutdown_connections 2024-04-25T00:00:35.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.736+0000 7fb167607700 1 -- 172.21.15.110:0/1317318197 wait complete. 2024-04-25T00:00:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb167607700 1 Processor -- start 2024-04-25T00:00:35.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb167607700 1 -- start start 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb167607700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160101160 0x7fb160196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb167607700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb160101b20 0x7fb160196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb167607700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1601071e0 0x7fb16019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb167607700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb160110260 con 0x7fb1601071e0 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb167607700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601100e0 con 0x7fb160101b20 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb167607700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601103e0 con 0x7fb160101160 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb165ba4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1601071e0 0x7fb16019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb165ba4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1601071e0 0x7fb16019af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56270/0 (socket says 172.21.15.110:56270) 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb165ba4700 1 -- 172.21.15.110:0/2504727460 learned_addr learned my addr 172.21.15.110:0/2504727460 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:35.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.737+0000 7fb164ba2700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb160101b20 0x7fb160196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb1653a3700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160101160 0x7fb160196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb165ba4700 1 -- 172.21.15.110:0/2504727460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160101160 msgr2=0x7fb160196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb165ba4700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb160101160 0x7fb160196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb165ba4700 1 -- 172.21.15.110:0/2504727460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb160101b20 msgr2=0x7fb160196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb165ba4700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb160101b20 0x7fb160196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb165ba4700 1 -- 172.21.15.110:0/2504727460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16019b680 con 0x7fb1601071e0 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb165ba4700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1601071e0 0x7fb16019af40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b830 tx=0x7fb158004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158021070 con 0x7fb1601071e0 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16019b910 con 0x7fb1601071e0 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb158007b80 con 0x7fb1601071e0 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580052a0 con 0x7fb1601071e0 2024-04-25T00:00:35.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.738+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1601a31f0 con 0x7fb1601071e0 2024-04-25T00:00:35.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.739+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb160062940 con 0x7fb1601071e0 2024-04-25T00:00:35.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.741+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb158024070 con 0x7fb1601071e0 2024-04-25T00:00:35.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.741+0000 7fb1567fc700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb14c0a2d30 0x7fb14c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.741+0000 7fb1653a3700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb14c0a2d30 0x7fb14c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.742+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb15800f020 con 0x7fb1601071e0 2024-04-25T00:00:35.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.742+0000 7fb1567fc700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] conn(0x7fb14c0ac510 0x7fb14c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:35.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.742+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 --> [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb14c0aefe0 con 0x7fb14c0ac510 2024-04-25T00:00:35.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.742+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb1580d0c70 con 0x7fb1601071e0 2024-04-25T00:00:35.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.742+0000 7fb164ba2700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] conn(0x7fb14c0ac510 0x7fb14c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:35.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.742+0000 7fb1653a3700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb14c0a2d30 0x7fb14c0a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00dad0 tx=0x7fb15c00d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.742+0000 7fb164ba2700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] conn(0x7fb14c0ac510 0x7fb14c0ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:35.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.743+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 <== osd.3 v2:172.21.15.110:6810/2506639922 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb14c0aefe0 con 0x7fb14c0ac510 2024-04-25T00:00:35.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.759+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 --> [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb160062d40 con 0x7fb14c0ac510 2024-04-25T00:00:35.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.759+0000 7fb1567fc700 1 -- 172.21.15.110:0/2504727460 <== osd.3 v2:172.21.15.110:6810/2506639922 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb160062d40 con 0x7fb14c0ac510 2024-04-25T00:00:35.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] conn(0x7fb14c0ac510 msgr2=0x7fb14c0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] conn(0x7fb14c0ac510 0x7fb14c0ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb14c0a2d30 msgr2=0x7fb14c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb14c0a2d30 0x7fb14c0a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00dad0 tx=0x7fb15c00d4b0 comp rx=0 tx=0).stop 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1601071e0 msgr2=0x7fb16019af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 --2- 172.21.15.110:0/2504727460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1601071e0 0x7fb16019af40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b830 tx=0x7fb158004d00 comp rx=0 tx=0).stop 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb165ba4700 1 -- 172.21.15.110:0/2504727460 reap_dead start 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 shutdown_connections 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 >> 172.21.15.110:0/2504727460 conn(0x7fb1600771a0 msgr2=0x7fb1600fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 shutdown_connections 2024-04-25T00:00:35.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:35.760+0000 7fb167607700 1 -- 172.21.15.110:0/2504727460 wait complete. 2024-04-25T00:00:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:35 smithi188 ceph-mon[33285]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T00:00:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:35 smithi173 ceph-mon[32402]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T00:00:37.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 -- 172.21.15.110:0/2375010072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20100780 msgr2=0x7f9a20100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 --2- 172.21.15.110:0/2375010072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20100780 0x7f9a20100c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9a100099c0 tx=0x7f9a10009cd0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 -- 172.21.15.110:0/2375010072 shutdown_connections 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 --2- 172.21.15.110:0/2375010072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a20101140 0x7f9a2010f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 --2- 172.21.15.110:0/2375010072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a20107250 0x7f9a20107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 --2- 172.21.15.110:0/2375010072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20100780 0x7f9a20100c00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 -- 172.21.15.110:0/2375010072 >> 172.21.15.110:0/2375010072 conn(0x7f9a20077730 msgr2=0x7f9a200feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 -- 172.21.15.110:0/2375010072 shutdown_connections 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.614+0000 7f9a279d2700 1 -- 172.21.15.110:0/2375010072 wait complete. 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a279d2700 1 Processor -- start 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a279d2700 1 -- start start 2024-04-25T00:00:37.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a279d2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a20100780 0x7f9a20101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a279d2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20101140 0x7f9a20101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a279d2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a20107250 0x7f9a201023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a279d2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9a201125e0 con 0x7f9a20101140 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a279d2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20112460 con 0x7f9a20100780 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a279d2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20112760 con 0x7f9a20107250 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a24f6d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20101140 0x7f9a20101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a24f6d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20101140 0x7f9a20101ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48004/0 (socket says 172.21.15.110:48004) 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.615+0000 7f9a24f6d700 1 -- 172.21.15.110:0/1390684169 learned_addr learned my addr 172.21.15.110:0/1390684169 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a24f6d700 1 -- 172.21.15.110:0/1390684169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a20107250 msgr2=0x7f9a201023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a24f6d700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a20107250 0x7f9a201023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a24f6d700 1 -- 172.21.15.110:0/1390684169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a20100780 msgr2=0x7f9a20101960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a24f6d700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a20100780 0x7f9a20101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a24f6d700 1 -- 172.21.15.110:0/1390684169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a201a6f70 con 0x7f9a20101140 2024-04-25T00:00:37.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a24f6d700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20101140 0x7f9a20101ea0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009fd0 tx=0x7f9a100037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10027070 con 0x7f9a20101140 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a10004140 con 0x7f9a20101140 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10022e60 con 0x7f9a20101140 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a201a7200 con 0x7f9a20101140 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.616+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201a76b0 con 0x7f9a20101140 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.617+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a20062940 con 0x7f9a20101140 2024-04-25T00:00:37.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.619+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9a1002a070 con 0x7f9a20101140 2024-04-25T00:00:37.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.620+0000 7f9a167fc700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9a0c0a2db0 0x7f9a0c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.620+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9a100d0730 con 0x7f9a20101140 2024-04-25T00:00:37.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.620+0000 7f9a2576e700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9a0c0a2db0 0x7f9a0c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.621+0000 7f9a167fc700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] conn(0x7f9a0c0ac590 0x7f9a0c0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:37.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.621+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 --> [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a10010ea0 con 0x7f9a0c0ac590 2024-04-25T00:00:37.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.621+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f9a10099210 con 0x7f9a20101140 2024-04-25T00:00:37.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.621+0000 7f9a25f6f700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] conn(0x7f9a0c0ac590 0x7f9a0c0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:37.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.621+0000 7f9a2576e700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9a0c0a2db0 0x7f9a0c0a5270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0097b0 tx=0x7f9a1c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.622+0000 7f9a25f6f700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] conn(0x7f9a0c0ac590 0x7f9a0c0ae9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:37.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.622+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 <== osd.8 v2:172.21.15.188:6816/879002066 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a10010ea0 con 0x7f9a0c0ac590 2024-04-25T00:00:37.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.640+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 --> [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a20062d40 con 0x7f9a0c0ac590 2024-04-25T00:00:37.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a167fc700 1 -- 172.21.15.110:0/1390684169 <== osd.8 v2:172.21.15.188:6816/879002066 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9a20062d40 con 0x7f9a0c0ac590 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 >> [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] conn(0x7f9a0c0ac590 msgr2=0x7f9a0c0ae9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] conn(0x7f9a0c0ac590 0x7f9a0c0ae9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9a0c0a2db0 msgr2=0x7f9a0c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9a0c0a2db0 0x7f9a0c0a5270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0097b0 tx=0x7f9a1c006d20 comp rx=0 tx=0).stop 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20101140 msgr2=0x7f9a20101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 --2- 172.21.15.110:0/1390684169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a20101140 0x7f9a20101ea0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009fd0 tx=0x7f9a100037a0 comp rx=0 tx=0).stop 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a25f6f700 1 -- 172.21.15.110:0/1390684169 reap_dead start 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 shutdown_connections 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 >> 172.21.15.110:0/1390684169 conn(0x7f9a20077730 msgr2=0x7f9a201053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 shutdown_connections 2024-04-25T00:00:37.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:37.641+0000 7f9a279d2700 1 -- 172.21.15.110:0/1390684169 wait complete. 2024-04-25T00:00:37.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:37 smithi110 ceph-mon[29187]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:00:37.904 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:37 smithi173 ceph-mon[32402]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:00:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:37 smithi188 ceph-mon[33285]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:00:38.697 INFO:teuthology.orchestra.run.smithi110.stdout:34359738443 2024-04-25T00:00:38.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T00:00:40.003 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:39 smithi110 ceph-mon[29187]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:00:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:39 smithi188 ceph-mon[33285]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:00:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:39 smithi173 ceph-mon[32402]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:00:40.871 INFO:teuthology.orchestra.run.smithi110.stdout:141733920824 2024-04-25T00:00:40.872 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T00:00:41.650 INFO:teuthology.orchestra.run.smithi110.stdout:55834574918 2024-04-25T00:00:41.650 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T00:00:41.934 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:41 smithi110 ceph-mon[29187]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:00:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:41 smithi188 ceph-mon[33285]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:00:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:41 smithi173 ceph-mon[32402]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.676+0000 7f4fcb66e700 1 -- 172.21.15.110:0/659619146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4102540 msgr2=0x7f4fc4105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.676+0000 7f4fcb66e700 1 --2- 172.21.15.110:0/659619146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4102540 0x7f4fc4105dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40099c0 tx=0x7f4fb4009cd0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 -- 172.21.15.110:0/659619146 shutdown_connections 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 --2- 172.21.15.110:0/659619146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc41064b0 0x7f4fc410e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 --2- 172.21.15.110:0/659619146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc4101b90 0x7f4fc4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 --2- 172.21.15.110:0/659619146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4102540 0x7f4fc4105dd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 -- 172.21.15.110:0/659619146 >> 172.21.15.110:0/659619146 conn(0x7f4fc40777a0 msgr2=0x7f4fc40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 -- 172.21.15.110:0/659619146 shutdown_connections 2024-04-25T00:00:42.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 -- 172.21.15.110:0/659619146 wait complete. 2024-04-25T00:00:42.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 Processor -- start 2024-04-25T00:00:42.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.677+0000 7f4fcb66e700 1 -- start start 2024-04-25T00:00:42.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fcb66e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4101b90 0x7f4fc419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fcb66e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc4102540 0x7f4fc419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fcb66e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc41064b0 0x7f4fc4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fcb66e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4111460 con 0x7f4fc4101b90 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fcb66e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41112e0 con 0x7f4fc41064b0 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fcb66e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41115e0 con 0x7f4fc4102540 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4101b90 0x7f4fc419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4101b90 0x7f4fc419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48028/0 (socket says 172.21.15.110:48028) 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 -- 172.21.15.110:0/3352900691 learned_addr learned my addr 172.21.15.110:0/3352900691 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc9c0b700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc41064b0 0x7f4fc4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc8c09700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc4102540 0x7f4fc419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 -- 172.21.15.110:0/3352900691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc4102540 msgr2=0x7f4fc419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc4102540 0x7f4fc419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 -- 172.21.15.110:0/3352900691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc41064b0 msgr2=0x7f4fc4197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc41064b0 0x7f4fc4197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 -- 172.21.15.110:0/3352900691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc4197f60 con 0x7f4fc4101b90 2024-04-25T00:00:42.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fc940a700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4101b90 0x7f4fc419d600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000b530 tx=0x7f4fc000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0015020 con 0x7f4fc4101b90 2024-04-25T00:00:42.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fc0011ce0 con 0x7f4fc4101b90 2024-04-25T00:00:42.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0010d60 con 0x7f4fc4101b90 2024-04-25T00:00:42.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.678+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc4198250 con 0x7f4fc4101b90 2024-04-25T00:00:42.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.679+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc4100820 con 0x7f4fc4101b90 2024-04-25T00:00:42.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.679+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4fc4100df0 con 0x7f4fc4101b90 2024-04-25T00:00:42.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.680+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4fc0018020 con 0x7f4fc4101b90 2024-04-25T00:00:42.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.681+0000 7f4fba7fc700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4fb00a2cb0 0x7f4fb00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.681+0000 7f4fc8c09700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4fb00a2cb0 0x7f4fb00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.681+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4fc00cee70 con 0x7f4fc4101b90 2024-04-25T00:00:42.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.681+0000 7f4fba7fc700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] conn(0x7f4fb00ac490 0x7f4fb00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:42.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.681+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 --> [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4fc0018ec0 con 0x7f4fb00ac490 2024-04-25T00:00:42.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.681+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f4fc00cf350 con 0x7f4fc4101b90 2024-04-25T00:00:42.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.681+0000 7f4fc8c09700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4fb00a2cb0 0x7f4fb00a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.682+0000 7f4fc9c0b700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] conn(0x7f4fb00ac490 0x7f4fb00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:42.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.682+0000 7f4fc9c0b700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] conn(0x7f4fb00ac490 0x7f4fb00ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:42.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.683+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 <== osd.2 v2:172.21.15.188:6800/2140926702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4fc0018ec0 con 0x7f4fb00ac490 2024-04-25T00:00:42.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.699+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 --> [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4fc4101270 con 0x7f4fb00ac490 2024-04-25T00:00:42.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fba7fc700 1 -- 172.21.15.110:0/3352900691 <== osd.2 v2:172.21.15.188:6800/2140926702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4fc4101270 con 0x7f4fb00ac490 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 >> [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] conn(0x7f4fb00ac490 msgr2=0x7f4fb00ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] conn(0x7f4fb00ac490 0x7f4fb00ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4fb00a2cb0 msgr2=0x7f4fb00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4fb00a2cb0 0x7f4fb00a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4005fd0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4101b90 msgr2=0x7f4fc419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 --2- 172.21.15.110:0/3352900691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fc4101b90 0x7f4fc419d600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000b530 tx=0x7f4fc000b8f0 comp rx=0 tx=0).stop 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fc9c0b700 1 -- 172.21.15.110:0/3352900691 reap_dead start 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 shutdown_connections 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 >> 172.21.15.110:0/3352900691 conn(0x7f4fc40777a0 msgr2=0x7f4fc40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.700+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 shutdown_connections 2024-04-25T00:00:42.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.701+0000 7f4fcb66e700 1 -- 172.21.15.110:0/3352900691 wait complete. 2024-04-25T00:00:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.036+0000 7fb2dc842700 1 -- 172.21.15.110:0/1786820356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 msgr2=0x7fb2d810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.036+0000 7fb2dc842700 1 --2- 172.21.15.110:0/1786820356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 0x7fb2d810e020 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80099c0 tx=0x7fb2c8009cd0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 -- 172.21.15.110:0/1786820356 shutdown_connections 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 --2- 172.21.15.110:0/1786820356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 0x7fb2d810e020 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 --2- 172.21.15.110:0/1786820356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2d81013c0 0x7fb2d81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 --2- 172.21.15.110:0/1786820356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2d8101d70 0x7fb2d8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 -- 172.21.15.110:0/1786820356 >> 172.21.15.110:0/1786820356 conn(0x7fb2d80780e0 msgr2=0x7fb2d80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 -- 172.21.15.110:0/1786820356 shutdown_connections 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 -- 172.21.15.110:0/1786820356 wait complete. 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 Processor -- start 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.037+0000 7fb2dc842700 1 -- start start 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2dc842700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2d81013c0 0x7fb2d8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2dc842700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2d8101d70 0x7fb2d8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2dc842700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 0x7fb2d81a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2dc842700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81112f0 con 0x7fb2d8106450 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2dc842700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8111170 con 0x7fb2d8101d70 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2dc842700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8111470 con 0x7fb2d81013c0 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 0x7fb2d81a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 0x7fb2d81a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48024/0 (socket says 172.21.15.110:48024) 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 -- 172.21.15.110:0/475773859 learned_addr learned my addr 172.21.15.110:0/475773859 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d5d9b700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2d8101d70 0x7fb2d8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d659c700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2d81013c0 0x7fb2d8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 -- 172.21.15.110:0/475773859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2d81013c0 msgr2=0x7fb2d8199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2d81013c0 0x7fb2d8199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 -- 172.21.15.110:0/475773859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2d8101d70 msgr2=0x7fb2d8199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2d8101d70 0x7fb2d8199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 -- 172.21.15.110:0/475773859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d81a4d30 con 0x7fb2d8106450 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2d6d9d700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 0x7fb2d81a47f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80059d0 tx=0x7fb2c80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8026070 con 0x7fb2d8106450 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d81a4f00 con 0x7fb2d8106450 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2c800f460 con 0x7fb2d8106450 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8009e60 con 0x7fb2d8106450 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.038+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81a5300 con 0x7fb2d8106450 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.039+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2d80628c0 con 0x7fb2d8106450 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.040+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb2c8029070 con 0x7fb2d8106450 2024-04-25T00:00:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.041+0000 7fb2cf7fe700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb2c40a2cb0 0x7fb2c40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.041+0000 7fb2d659c700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb2c40a2cb0 0x7fb2c40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.041+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb2c80cfcc0 con 0x7fb2d8106450 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.041+0000 7fb2cf7fe700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] conn(0x7fb2c40ac490 0x7fb2c40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.041+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 --> [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2c8001d30 con 0x7fb2c40ac490 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.041+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb2c80d01a0 con 0x7fb2d8106450 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.041+0000 7fb2d5d9b700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] conn(0x7fb2c40ac490 0x7fb2c40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.041+0000 7fb2d659c700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb2c40a2cb0 0x7fb2c40a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009380 tx=0x7fb2d000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.042+0000 7fb2d5d9b700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] conn(0x7fb2c40ac490 0x7fb2c40ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.042+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 <== osd.6 v2:172.21.15.110:6818/3643107507 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2c8001d30 con 0x7fb2c40ac490 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.058+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 --> [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb2d8062cc0 con 0x7fb2c40ac490 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2cf7fe700 1 -- 172.21.15.110:0/475773859 <== osd.6 v2:172.21.15.110:6818/3643107507 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb2d8062cc0 con 0x7fb2c40ac490 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] conn(0x7fb2c40ac490 msgr2=0x7fb2c40ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] conn(0x7fb2c40ac490 0x7fb2c40ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb2c40a2cb0 msgr2=0x7fb2c40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fb2c40a2cb0 0x7fb2c40a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009380 tx=0x7fb2d000b040 comp rx=0 tx=0).stop 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 msgr2=0x7fb2d81a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 --2- 172.21.15.110:0/475773859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2d8106450 0x7fb2d81a47f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80059d0 tx=0x7fb2c80045e0 comp rx=0 tx=0).stop 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2d6d9d700 1 -- 172.21.15.110:0/475773859 reap_dead start 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 shutdown_connections 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 >> 172.21.15.110:0/475773859 conn(0x7fb2d80780e0 msgr2=0x7fb2d80ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 shutdown_connections 2024-04-25T00:00:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:42.059+0000 7fb2dc842700 1 -- 172.21.15.110:0/475773859 wait complete. 2024-04-25T00:00:43.888 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:43 smithi110 ceph-mon[29187]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:00:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:43 smithi188 ceph-mon[33285]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:00:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:43 smithi173 ceph-mon[32402]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:00:45.126 INFO:teuthology.orchestra.run.smithi110.stdout:107374182461 2024-04-25T00:00:45.127 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T00:00:45.776 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:45 smithi110 ceph-mon[29187]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:00:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:45 smithi188 ceph-mon[33285]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:00:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:45 smithi173 ceph-mon[32402]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:00:46.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.118+0000 7f45f4916700 1 -- 172.21.15.110:0/2061580002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f006a980 msgr2=0x7f45f006ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.118+0000 7f45f4916700 1 --2- 172.21.15.110:0/2061580002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f006a980 0x7f45f006ade0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009990 tx=0x7f45e0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 -- 172.21.15.110:0/2061580002 shutdown_connections 2024-04-25T00:00:46.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 --2- 172.21.15.110:0/2061580002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45f006b320 0x7f45f010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 --2- 172.21.15.110:0/2061580002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f01082b0 0x7f45f0108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 --2- 172.21.15.110:0/2061580002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f006a980 0x7f45f006ade0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 -- 172.21.15.110:0/2061580002 >> 172.21.15.110:0/2061580002 conn(0x7f45f0078100 msgr2=0x7f45f0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 -- 172.21.15.110:0/2061580002 shutdown_connections 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 -- 172.21.15.110:0/2061580002 wait complete. 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 Processor -- start 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.119+0000 7f45f4916700 1 -- start start 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45f4916700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45f006a980 0x7f45f019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45f4916700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f006b320 0x7f45f019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45f4916700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f01082b0 0x7f45f0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45f4916700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f45f01123c0 con 0x7f45f01082b0 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45f4916700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f45f0112240 con 0x7f45f006a980 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45f4916700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45f0112540 con 0x7f45f006b320 2024-04-25T00:00:46.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f01082b0 0x7f45f0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f01082b0 0x7f45f0198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48076/0 (socket says 172.21.15.110:48076) 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 -- 172.21.15.110:0/2905137333 learned_addr learned my addr 172.21.15.110:0/2905137333 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45ee59c700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45f006a980 0x7f45f019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45edd9b700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f006b320 0x7f45f019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 -- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f006b320 msgr2=0x7f45f019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f006b320 0x7f45f019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 -- 172.21.15.110:0/2905137333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45f006a980 msgr2=0x7f45f019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45f006a980 0x7f45f019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 -- 172.21.15.110:0/2905137333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f0198f60 con 0x7f45f01082b0 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45edd9b700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f006b320 0x7f45f019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45eed9d700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f01082b0 0x7f45f0198780 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45e800ada0 tx=0x7f45e800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8017020 con 0x7f45f01082b0 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f0199250 con 0x7f45f01082b0 2024-04-25T00:00:46.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45e800ce30 con 0x7f45f01082b0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8005350 con 0x7f45f01082b0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.120+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f01a5460 con 0x7f45f01082b0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.121+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45f00629c0 con 0x7f45f01082b0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.123+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f45e800c980 con 0x7f45f01082b0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.123+0000 7f45df7fe700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f45d80a2cb0 0x7f45d80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.123+0000 7f45ee59c700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f45d80a2cb0 0x7f45d80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.123+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f45e80cf100 con 0x7f45f01082b0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.123+0000 7f45df7fe700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] conn(0x7f45d80ac490 0x7f45d80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.123+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 --> [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45e8022e50 con 0x7f45d80ac490 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.123+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f45e80cf5e0 con 0x7f45f01082b0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.124+0000 7f45ee59c700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f45d80a2cb0 0x7f45d80a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45e40045b0 tx=0x7f45e400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.124+0000 7f45edd9b700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] conn(0x7f45d80ac490 0x7f45d80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.124+0000 7f45edd9b700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] conn(0x7f45d80ac490 0x7f45d80ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.125+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 <== osd.5 v2:172.21.15.188:6808/216258077 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45e8022e50 con 0x7f45d80ac490 2024-04-25T00:00:46.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.141+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 --> [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45f0062dc0 con 0x7f45d80ac490 2024-04-25T00:00:46.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.142+0000 7f45df7fe700 1 -- 172.21.15.110:0/2905137333 <== osd.5 v2:172.21.15.188:6808/216258077 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f45f0062dc0 con 0x7f45d80ac490 2024-04-25T00:00:46.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] conn(0x7f45d80ac490 msgr2=0x7f45d80ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] conn(0x7f45d80ac490 0x7f45d80ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f45d80a2cb0 msgr2=0x7f45d80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f45d80a2cb0 0x7f45d80a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45e40045b0 tx=0x7f45e400b3c0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f01082b0 msgr2=0x7f45f0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 --2- 172.21.15.110:0/2905137333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45f01082b0 0x7f45f0198780 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45e800ada0 tx=0x7f45e800c330 comp rx=0 tx=0).stop 2024-04-25T00:00:46.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45eed9d700 1 -- 172.21.15.110:0/2905137333 reap_dead start 2024-04-25T00:00:46.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 shutdown_connections 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 >> 172.21.15.110:0/2905137333 conn(0x7f45f0078100 msgr2=0x7f45f0100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 shutdown_connections 2024-04-25T00:00:46.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.143+0000 7f45f4916700 1 -- 172.21.15.110:0/2905137333 wait complete. 2024-04-25T00:00:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/901291639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 msgr2=0x7f9bc410f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 --2- 172.21.15.110:0/901291639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 0x7f9bc410f170 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00b0a0 tx=0x7f9bbc00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/901291639 shutdown_connections 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 --2- 172.21.15.110:0/901291639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 0x7f9bc410f170 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 --2- 172.21.15.110:0/901291639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc41024e0 0x7f9bc41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 --2- 172.21.15.110:0/901291639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bc4102e90 0x7f9bc4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/901291639 >> 172.21.15.110:0/901291639 conn(0x7f9bc40793e0 msgr2=0x7f9bc40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/901291639 shutdown_connections 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.548+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/901291639 wait complete. 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bcaf5e700 1 Processor -- start 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bcaf5e700 1 -- start start 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bcaf5e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc41024e0 0x7f9bc41a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bcaf5e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bc4102e90 0x7f9bc41a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bcaf5e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 0x7f9bc419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bcaf5e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4112430 con 0x7f9bc41075a0 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bcaf5e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc41122b0 con 0x7f9bc4102e90 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bcaf5e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc41125b0 con 0x7f9bc41024e0 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 0x7f9bc419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 0x7f9bc419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52324/0 (socket says 172.21.15.110:52324) 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 -- 172.21.15.110:0/2086951420 learned_addr learned my addr 172.21.15.110:0/2086951420 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc8cfa700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc41024e0 0x7f9bc41a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc3fff700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bc4102e90 0x7f9bc41a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 -- 172.21.15.110:0/2086951420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc41024e0 msgr2=0x7f9bc41a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc41024e0 0x7f9bc41a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 -- 172.21.15.110:0/2086951420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bc4102e90 msgr2=0x7f9bc41a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bc4102e90 0x7f9bc41a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 -- 172.21.15.110:0/2086951420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc419b050 con 0x7f9bc41075a0 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc94fb700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 0x7f9bc419a8f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00b780 tx=0x7f9bbc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.549+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc021070 con 0x7f9bc41075a0 2024-04-25T00:00:46.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.550+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bbc007e30 con 0x7f9bc41075a0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.550+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc004800 con 0x7f9bc41075a0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.550+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc419b2e0 con 0x7f9bc41075a0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.550+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc41a7600 con 0x7f9bc41075a0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.550+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9bc40757c0 con 0x7f9bc41075a0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.552+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9bbc007980 con 0x7f9bc41075a0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.552+0000 7f9bc1ffb700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9bac0a2d30 0x7f9bac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.552+0000 7f9bc8cfa700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9bac0a2d30 0x7f9bac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.552+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9bbc00f020 con 0x7f9bc41075a0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.553+0000 7f9bc1ffb700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] conn(0x7f9bac0ac510 0x7f9bac0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.553+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 --> [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bbc02ee50 con 0x7f9bac0ac510 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.553+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f9bbc0d0b70 con 0x7f9bc41075a0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.553+0000 7f9bc8cfa700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9bac0a2d30 0x7f9bac0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005950 tx=0x7f9bb40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.553+0000 7f9bc3fff700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] conn(0x7f9bac0ac510 0x7f9bac0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:46.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.553+0000 7f9bc3fff700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] conn(0x7f9bac0ac510 0x7f9bac0ae930 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:46.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.554+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 <== osd.11 v2:172.21.15.188:6824/785596824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bbc02ee50 con 0x7f9bac0ac510 2024-04-25T00:00:46.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.570+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 --> [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9bc41039a0 con 0x7f9bac0ac510 2024-04-25T00:00:46.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bc1ffb700 1 -- 172.21.15.110:0/2086951420 <== osd.11 v2:172.21.15.188:6824/785596824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9bc41039a0 con 0x7f9bac0ac510 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 >> [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] conn(0x7f9bac0ac510 msgr2=0x7f9bac0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] conn(0x7f9bac0ac510 0x7f9bac0ae930 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9bac0a2d30 msgr2=0x7f9bac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f9bac0a2d30 0x7f9bac0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005950 tx=0x7f9bb40058e0 comp rx=0 tx=0).stop 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 msgr2=0x7f9bc419a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 --2- 172.21.15.110:0/2086951420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bc41075a0 0x7f9bc419a8f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00b780 tx=0x7f9bbc004d00 comp rx=0 tx=0).stop 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bc94fb700 1 -- 172.21.15.110:0/2086951420 reap_dead start 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 shutdown_connections 2024-04-25T00:00:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 >> 172.21.15.110:0/2086951420 conn(0x7f9bc40793e0 msgr2=0x7f9bc4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:46.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 shutdown_connections 2024-04-25T00:00:46.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:46.571+0000 7f9bcaf5e700 1 -- 172.21.15.110:0/2086951420 wait complete. 2024-04-25T00:00:47.328 INFO:teuthology.orchestra.run.smithi110.stdout:197568495662 2024-04-25T00:00:47.328 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T00:00:47.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 -- 172.21.15.110:0/2080638166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101130 msgr2=0x7f8ee41015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 --2- 172.21.15.110:0/2080638166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101130 0x7f8ee41015b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009990 tx=0x7f8ed0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 -- 172.21.15.110:0/2080638166 shutdown_connections 2024-04-25T00:00:47.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 --2- 172.21.15.110:0/2080638166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4101af0 0x7f8ee410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 --2- 172.21.15.110:0/2080638166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee41071a0 0x7f8ee4107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 --2- 172.21.15.110:0/2080638166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101130 0x7f8ee41015b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 -- 172.21.15.110:0/2080638166 >> 172.21.15.110:0/2080638166 conn(0x7f8ee40780e0 msgr2=0x7f8ee40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 -- 172.21.15.110:0/2080638166 shutdown_connections 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.870+0000 7f8eeb007700 1 -- 172.21.15.110:0/2080638166 wait complete. 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8eeb007700 1 Processor -- start 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8eeb007700 1 -- start start 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8eeb007700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4101130 0x7f8ee419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8eeb007700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101af0 0x7f8ee419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8eeb007700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee41071a0 0x7f8ee4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8eeb007700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4111340 con 0x7f8ee4101af0 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8eeb007700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee41111c0 con 0x7f8ee41071a0 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8eeb007700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee41114c0 con 0x7f8ee4101130 2024-04-25T00:00:47.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101af0 0x7f8ee419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101af0 0x7f8ee419db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48052/0 (socket says 172.21.15.110:48052) 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 -- 172.21.15.110:0/2701708581 learned_addr learned my addr 172.21.15.110:0/2701708581 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee8da3700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4101130 0x7f8ee419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 -- 172.21.15.110:0/2701708581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4101130 msgr2=0x7f8ee419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee4101130 0x7f8ee419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee95a4700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee41071a0 0x7f8ee4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 -- 172.21.15.110:0/2701708581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee41071a0 msgr2=0x7f8ee4197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee41071a0 0x7f8ee4197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 -- 172.21.15.110:0/2701708581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee4197f20 con 0x7f8ee4101af0 2024-04-25T00:00:47.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.871+0000 7f8ee3fff700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101af0 0x7f8ee419db10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0003ab0 tx=0x7f8ed0003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.872+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0027070 con 0x7f8ee4101af0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.872+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed000bdc0 con 0x7f8ee4101af0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.872+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee41981b0 con 0x7f8ee4101af0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.872+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0022de0 con 0x7f8ee4101af0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.872+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee41a43f0 con 0x7f8ee4101af0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.872+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ee4062940 con 0x7f8ee4101af0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.874+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8ed002a070 con 0x7f8ee4101af0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.874+0000 7f8ee1ffb700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ed40a2cb0 0x7f8ed40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.874+0000 7f8ee8da3700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ed40a2cb0 0x7f8ed40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.875+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8ed00d05f0 con 0x7f8ee4101af0 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.875+0000 7f8ee1ffb700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] conn(0x7f8ed40ac490 0x7f8ed40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.875+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 --> [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ed0010ec0 con 0x7f8ed40ac490 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.875+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8ed00d0ad0 con 0x7f8ee4101af0 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.875+0000 7f8ee8da3700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ed40a2cb0 0x7f8ed40a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8006fd0 tx=0x7f8ed8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.875+0000 7f8ee95a4700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] conn(0x7f8ed40ac490 0x7f8ed40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.875+0000 7f8ee95a4700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] conn(0x7f8ed40ac490 0x7f8ed40ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.876+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 <== osd.10 v2:172.21.15.173:6824/3803887034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ed0010ec0 con 0x7f8ed40ac490 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.893+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 --> [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ee4062d40 con 0x7f8ed40ac490 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.893+0000 7f8ee1ffb700 1 -- 172.21.15.110:0/2701708581 <== osd.10 v2:172.21.15.173:6824/3803887034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8ee4062d40 con 0x7f8ed40ac490 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 >> [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] conn(0x7f8ed40ac490 msgr2=0x7f8ed40ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] conn(0x7f8ed40ac490 0x7f8ed40ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ed40a2cb0 msgr2=0x7f8ed40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8ed40a2cb0 0x7f8ed40a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8006fd0 tx=0x7f8ed8008040 comp rx=0 tx=0).stop 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101af0 msgr2=0x7f8ee419db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 --2- 172.21.15.110:0/2701708581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ee4101af0 0x7f8ee419db10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0003ab0 tx=0x7f8ed0003ae0 comp rx=0 tx=0).stop 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8ee95a4700 1 -- 172.21.15.110:0/2701708581 reap_dead start 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 shutdown_connections 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 >> 172.21.15.110:0/2701708581 conn(0x7f8ee40780e0 msgr2=0x7f8ee40ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 shutdown_connections 2024-04-25T00:00:47.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:43.894+0000 7f8eeb007700 1 -- 172.21.15.110:0/2701708581 wait complete. 2024-04-25T00:00:48.001 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:47 smithi110 ceph-mon[29187]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:47 smithi188 ceph-mon[33285]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:48.028 INFO:teuthology.orchestra.run.smithi110.stdout:309237645339 2024-04-25T00:00:48.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T00:00:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:47 smithi173 ceph-mon[32402]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:48.532 INFO:teuthology.orchestra.run.smithi110.stdout:163208757301 2024-04-25T00:00:48.533 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T00:00:49.073 INFO:teuthology.orchestra.run.smithi110.stdout:253403070499 2024-04-25T00:00:49.074 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T00:00:49.487 INFO:teuthology.orchestra.run.smithi110.stdout:343597383701 2024-04-25T00:00:49.488 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2348522597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101120 msgr2=0x7f56dc1015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 --2- 172.21.15.110:0/2348522597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101120 0x7f56dc1015a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009990 tx=0x7f56d0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2348522597 shutdown_connections 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 --2- 172.21.15.110:0/2348522597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56dc101ae0 0x7f56dc10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 --2- 172.21.15.110:0/2348522597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc107190 0x7f56dc107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 --2- 172.21.15.110:0/2348522597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101120 0x7f56dc1015a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2348522597 >> 172.21.15.110:0/2348522597 conn(0x7f56dc0780a0 msgr2=0x7f56dc0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2348522597 shutdown_connections 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.037+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2348522597 wait complete. 2024-04-25T00:00:49.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56e0bb1700 1 Processor -- start 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56e0bb1700 1 -- start start 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56e0bb1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc101120 0x7f56dc1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56e0bb1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101ae0 0x7f56dc1035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56e0bb1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56dc107190 0x7f56dc103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56e0bb1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1112d0 con 0x7f56dc101ae0 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56e0bb1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc111150 con 0x7f56dc107190 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56e0bb1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc111450 con 0x7f56dc101120 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56d9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101ae0 0x7f56dc1035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56d9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101ae0 0x7f56dc1035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52336/0 (socket says 172.21.15.110:52336) 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.038+0000 7f56d9d9b700 1 -- 172.21.15.110:0/2838297375 learned_addr learned my addr 172.21.15.110:0/2838297375 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56da59c700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc101120 0x7f56dc1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56dad9d700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56dc107190 0x7f56dc103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56d9d9b700 1 -- 172.21.15.110:0/2838297375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc101120 msgr2=0x7f56dc1030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56d9d9b700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56dc101120 0x7f56dc1030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56d9d9b700 1 -- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56dc107190 msgr2=0x7f56dc103b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56d9d9b700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56dc107190 0x7f56dc103b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56d9d9b700 1 -- 172.21.15.110:0/2838297375 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc1a6db0 con 0x7f56dc101ae0 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56dad9d700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56dc107190 0x7f56dc103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:00:49.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56d9d9b700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101ae0 0x7f56dc1035e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f56d0000c00 tx=0x7f56d0003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0027070 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc1a7040 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56d000bdc0 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0022d30 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.039+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc1a7490 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.040+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f56dc06aa70 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.042+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f56d002a070 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.043+0000 7f56cb7fe700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f56c40a2cb0 0x7f56c40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.043+0000 7f56da59c700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f56c40a2cb0 0x7f56c40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.043+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f56d00d03a0 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.043+0000 7f56cb7fe700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] conn(0x7f56c40ac490 0x7f56c40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.043+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 --> [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56d00095d0 con 0x7f56c40ac490 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.043+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f56d00d0880 con 0x7f56dc101ae0 2024-04-25T00:00:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.043+0000 7f56dad9d700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] conn(0x7f56c40ac490 0x7f56c40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.044+0000 7f56da59c700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f56c40a2cb0 0x7f56c40a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0045b0 tx=0x7f56cc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.044+0000 7f56dad9d700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] conn(0x7f56c40ac490 0x7f56c40ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.045+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 <== osd.7 v2:172.21.15.173:6816/3085868439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56d00095d0 con 0x7f56c40ac490 2024-04-25T00:00:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.063+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 --> [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f56dc102840 con 0x7f56c40ac490 2024-04-25T00:00:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56cb7fe700 1 -- 172.21.15.110:0/2838297375 <== osd.7 v2:172.21.15.173:6816/3085868439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f56dc102840 con 0x7f56c40ac490 2024-04-25T00:00:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] conn(0x7f56c40ac490 msgr2=0x7f56c40ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] conn(0x7f56c40ac490 0x7f56c40ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f56c40a2cb0 msgr2=0x7f56c40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f56c40a2cb0 0x7f56c40a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0045b0 tx=0x7f56cc00b3c0 comp rx=0 tx=0).stop 2024-04-25T00:00:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101ae0 msgr2=0x7f56dc1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 --2- 172.21.15.110:0/2838297375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56dc101ae0 0x7f56dc1035e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f56d0000c00 tx=0x7f56d0003a80 comp rx=0 tx=0).stop 2024-04-25T00:00:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56dad9d700 1 -- 172.21.15.110:0/2838297375 reap_dead start 2024-04-25T00:00:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 shutdown_connections 2024-04-25T00:00:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 >> 172.21.15.110:0/2838297375 conn(0x7f56dc0780a0 msgr2=0x7f56dc105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 shutdown_connections 2024-04-25T00:00:49.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:49.064+0000 7f56e0bb1700 1 -- 172.21.15.110:0/2838297375 wait complete. 2024-04-25T00:00:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:49 smithi188 ceph-mon[33285]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:00:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:49 smithi110 ceph-mon[29187]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:00:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:49 smithi173 ceph-mon[32402]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:00:50.476 INFO:teuthology.orchestra.run.smithi110.stdout:77309411396 2024-04-25T00:00:50.476 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T00:00:51.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 -- 172.21.15.110:0/2561926613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d41071d0 msgr2=0x7f75d41075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 --2- 172.21.15.110:0/2561926613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d41071d0 0x7f75d41075b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009990 tx=0x7f75c4009ca0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 -- 172.21.15.110:0/2561926613 shutdown_connections 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 --2- 172.21.15.110:0/2561926613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75d4101b20 0x7f75d410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 --2- 172.21.15.110:0/2561926613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d41071d0 0x7f75d41075b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 --2- 172.21.15.110:0/2561926613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75d4101160 0x7f75d41015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 -- 172.21.15.110:0/2561926613 >> 172.21.15.110:0/2561926613 conn(0x7f75d4078170 msgr2=0x7f75d4078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 -- 172.21.15.110:0/2561926613 shutdown_connections 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.426+0000 7f75d9489700 1 -- 172.21.15.110:0/2561926613 wait complete. 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.427+0000 7f75d9489700 1 Processor -- start 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.427+0000 7f75d9489700 1 -- start start 2024-04-25T00:00:51.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d9489700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75d4101160 0x7f75d41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d9489700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4101b20 0x7f75d4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d9489700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75d41071d0 0x7f75d41a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d9489700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4111440 con 0x7f75d4101b20 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d9489700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41112c0 con 0x7f75d4101160 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d9489700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41115c0 con 0x7f75d41071d0 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4101b20 0x7f75d4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4101b20 0x7f75d4199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52350/0 (socket says 172.21.15.110:52350) 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 -- 172.21.15.110:0/3862437640 learned_addr learned my addr 172.21.15.110:0/3862437640 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d2ffd700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75d4101160 0x7f75d41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d37fe700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75d41071d0 0x7f75d41a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 -- 172.21.15.110:0/3862437640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75d41071d0 msgr2=0x7f75d41a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75d41071d0 0x7f75d41a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 -- 172.21.15.110:0/3862437640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75d4101160 msgr2=0x7f75d41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75d4101160 0x7f75d41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 -- 172.21.15.110:0/3862437640 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d41a4f80 con 0x7f75d4101b20 2024-04-25T00:00:51.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d37fe700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75d41071d0 0x7f75d41a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:00:51.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d27fc700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4101b20 0x7f75d4199f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b530 tx=0x7f75c800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8015020 con 0x7f75d4101b20 2024-04-25T00:00:51.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d41a51b0 con 0x7f75d4101b20 2024-04-25T00:00:51.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c8011ce0 con 0x7f75d4101b20 2024-04-25T00:00:51.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8010d60 con 0x7f75d4101b20 2024-04-25T00:00:51.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.428+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41a56f0 con 0x7f75d4101b20 2024-04-25T00:00:51.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.429+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75d4062940 con 0x7f75d4101b20 2024-04-25T00:00:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.430+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f75c8018020 con 0x7f75d4101b20 2024-04-25T00:00:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.431+0000 7f75bbfff700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f75bc0a2cb0 0x7f75bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.431+0000 7f75d2ffd700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f75bc0a2cb0 0x7f75bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.431+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f75c80cee30 con 0x7f75d4101b20 2024-04-25T00:00:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.431+0000 7f75bbfff700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] conn(0x7f75bc0ac490 0x7f75bc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.431+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 --> [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75c8018ec0 con 0x7f75bc0ac490 2024-04-25T00:00:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.431+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f75c80cf310 con 0x7f75d4101b20 2024-04-25T00:00:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.431+0000 7f75d37fe700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] conn(0x7f75bc0ac490 0x7f75bc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.431+0000 7f75d2ffd700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f75bc0a2cb0 0x7f75bc0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75c400b5c0 tx=0x7f75c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.432+0000 7f75d37fe700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] conn(0x7f75bc0ac490 0x7f75bc0ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.432+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 <== osd.9 v2:172.21.15.110:6826/453265858 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75c8018ec0 con 0x7f75bc0ac490 2024-04-25T00:00:51.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.448+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 --> [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75d4062d40 con 0x7f75bc0ac490 2024-04-25T00:00:51.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.449+0000 7f75bbfff700 1 -- 172.21.15.110:0/3862437640 <== osd.9 v2:172.21.15.110:6826/453265858 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f75d4062d40 con 0x7f75bc0ac490 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.449+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] conn(0x7f75bc0ac490 msgr2=0x7f75bc0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.449+0000 7f75d9489700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] conn(0x7f75bc0ac490 0x7f75bc0ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.449+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f75bc0a2cb0 msgr2=0x7f75bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.449+0000 7f75d9489700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f75bc0a2cb0 0x7f75bc0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75c400b5c0 tx=0x7f75c401a040 comp rx=0 tx=0).stop 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.449+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4101b20 msgr2=0x7f75d4199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.449+0000 7f75d9489700 1 --2- 172.21.15.110:0/3862437640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4101b20 0x7f75d4199f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b530 tx=0x7f75c800b8f0 comp rx=0 tx=0).stop 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.449+0000 7f75d37fe700 1 -- 172.21.15.110:0/3862437640 reap_dead start 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.450+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 shutdown_connections 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.450+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 >> 172.21.15.110:0/3862437640 conn(0x7f75d4078170 msgr2=0x7f75d40fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:51.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.450+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 shutdown_connections 2024-04-25T00:00:51.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:51.450+0000 7f75d9489700 1 -- 172.21.15.110:0/3862437640 wait complete. 2024-04-25T00:00:51.463 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:51.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:51 smithi110 ceph-mon[29187]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:51 smithi188 ceph-mon[33285]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:51 smithi173 ceph-mon[32402]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:52.147 INFO:teuthology.orchestra.run.smithi110.stdout:219043332139 2024-04-25T00:00:52.148 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T00:00:52.794 INFO:teuthology.orchestra.run.smithi110.stdout:287762808864 2024-04-25T00:00:52.795 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T00:00:53.917 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:53 smithi110 ceph-mon[29187]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:53 smithi188 ceph-mon[33285]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:53 smithi173 ceph-mon[32402]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:00:54.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 -- 172.21.15.110:0/3848342985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101af0 msgr2=0x7febe810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 --2- 172.21.15.110:0/3848342985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101af0 0x7febe810dfe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7febe000b0a0 tx=0x7febe000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 -- 172.21.15.110:0/3848342985 shutdown_connections 2024-04-25T00:00:54.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 --2- 172.21.15.110:0/3848342985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101af0 0x7febe810dfe0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 --2- 172.21.15.110:0/3848342985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe81071a0 0x7febe8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 --2- 172.21.15.110:0/3848342985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe8101130 0x7febe81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 -- 172.21.15.110:0/3848342985 >> 172.21.15.110:0/3848342985 conn(0x7febe80780e0 msgr2=0x7febe80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 -- 172.21.15.110:0/3848342985 shutdown_connections 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.263+0000 7febf07de700 1 -- 172.21.15.110:0/3848342985 wait complete. 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febf07de700 1 Processor -- start 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febf07de700 1 -- start start 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febf07de700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101130 0x7febe819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febf07de700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8101af0 0x7febe819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febf07de700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe81071a0 0x7febe8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febf07de700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7febe8111340 con 0x7febe8101130 2024-04-25T00:00:54.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febf07de700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7febe81111c0 con 0x7febe8101af0 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febf07de700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7febe81114c0 con 0x7febe81071a0 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febee57a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101130 0x7febe819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febee57a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101130 0x7febe819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52360/0 (socket says 172.21.15.110:52360) 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febee57a700 1 -- 172.21.15.110:0/1640336249 learned_addr learned my addr 172.21.15.110:0/1640336249 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febeed7b700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe81071a0 0x7febe8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.264+0000 7febedd79700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8101af0 0x7febe819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febee57a700 1 -- 172.21.15.110:0/1640336249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe81071a0 msgr2=0x7febe8197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febee57a700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe81071a0 0x7febe8197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febee57a700 1 -- 172.21.15.110:0/1640336249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8101af0 msgr2=0x7febe819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febee57a700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8101af0 0x7febe819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febee57a700 1 -- 172.21.15.110:0/1640336249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe8197d60 con 0x7febe8101130 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febee57a700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101130 0x7febe819d3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7febe4007b50 tx=0x7febe4007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febdf7fe700 1 -- 172.21.15.110:0/1640336249 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe400e500 con 0x7febe8101130 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe8198050 con 0x7febe8101130 2024-04-25T00:00:54.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febdf7fe700 1 -- 172.21.15.110:0/1640336249 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febe400eb50 con 0x7febe8101130 2024-04-25T00:00:54.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febdf7fe700 1 -- 172.21.15.110:0/1640336249 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe400aab0 con 0x7febe8101130 2024-04-25T00:00:54.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.265+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe8069f10 con 0x7febe8101130 2024-04-25T00:00:54.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.266+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe8068f20 con 0x7febe8101130 2024-04-25T00:00:54.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.267+0000 7febdf7fe700 1 -- 172.21.15.110:0/1640336249 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7febe400e6a0 con 0x7febe8101130 2024-04-25T00:00:54.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.268+0000 7febdf7fe700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7febd40a2cb0 0x7febd40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:00:54.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.268+0000 7febedd79700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7febd40a2cb0 0x7febd40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:00:54.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.268+0000 7febdf7fe700 1 -- 172.21.15.110:0/1640336249 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7febe40d0400 con 0x7febe8101130 2024-04-25T00:00:54.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.268+0000 7febedd79700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7febd40a2cb0 0x7febd40a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febd8006fd0 tx=0x7febd8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:00:54.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.272+0000 7febdf7fe700 1 -- 172.21.15.110:0/1640336249 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7febe4023060 con 0x7febe8101130 2024-04-25T00:00:54.323 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:54.325 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:54.327 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:54.333 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:54.333 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:54.335 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:54.336 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:00:54.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.533+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7febe806a910 con 0x7febe8101130 2024-04-25T00:00:54.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.533+0000 7febdf7fe700 1 -- 172.21.15.110:0/1640336249 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7febe409d440 con 0x7febe8101130 2024-04-25T00:00:54.533 INFO:teuthology.orchestra.run.smithi110.stdout:55834574922 2024-04-25T00:00:54.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7febd40a2cb0 msgr2=0x7febd40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7febd40a2cb0 0x7febd40a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febd8006fd0 tx=0x7febd8008040 comp rx=0 tx=0).stop 2024-04-25T00:00:54.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101130 msgr2=0x7febe819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:00:54.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101130 0x7febe819d3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7febe4007b50 tx=0x7febe4007e60 comp rx=0 tx=0).stop 2024-04-25T00:00:54.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 shutdown_connections 2024-04-25T00:00:54.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7febd40a2cb0 0x7febd40a5170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe81071a0 0x7febe8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febe8101130 0x7febe819d3f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 --2- 172.21.15.110:0/1640336249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8101af0 0x7febe819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:00:54.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 >> 172.21.15.110:0/1640336249 conn(0x7febe80780e0 msgr2=0x7febe80ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:00:54.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 shutdown_connections 2024-04-25T00:00:54.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:00:54.535+0000 7febf07de700 1 -- 172.21.15.110:0/1640336249 wait complete. 2024-04-25T00:00:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T00:00:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:00:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T00:00:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T00:00:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T00:00:54.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-25T00:00:54.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:00:54.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:54 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1640336249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:00:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T00:00:55.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:00:55.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T00:00:55.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T00:00:55.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T00:00:55.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-25T00:00:55.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:00:55.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:54 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1640336249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:00:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T00:00:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:00:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T00:00:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T00:00:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T00:00:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-25T00:00:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:00:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:54 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1640336249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:00:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:00:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T00:00:55.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:00:55.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T00:00:55.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T00:00:55.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T00:00:55.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-25T00:00:55.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:00:55.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:55 smithi110 ceph-mon[29187]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:00:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:00:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T00:00:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:00:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T00:00:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T00:00:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T00:00:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-25T00:00:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:00:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:55 smithi188 ceph-mon[33285]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:00:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:00:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T00:00:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:00:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T00:00:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T00:00:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T00:00:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-25T00:00:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:00:56.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:55 smithi173 ceph-mon[32402]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:00:56.350 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574922 for osd.1 2024-04-25T00:00:56.351 DEBUG:teuthology.parallel:result is None 2024-04-25T00:00:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:56 smithi188 ceph-mon[33285]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:00:57.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:56 smithi110 ceph-mon[29187]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:00:57.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:56 smithi173 ceph-mon[32402]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:00:57.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:57 smithi110 ceph-mon[29187]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T00:00:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:57 smithi188 ceph-mon[33285]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T00:00:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:57 smithi173 ceph-mon[32402]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T00:00:59.777 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:00:59 smithi110 ceph-mon[29187]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:00:59 smithi188 ceph-mon[33285]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:00:59 smithi173 ceph-mon[32402]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 -- 172.21.15.110:0/3609148265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb0069f40 msgr2=0x7f1eb006a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/3609148265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb0069f40 0x7f1eb006a3a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009990 tx=0x7f1e98009ca0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 -- 172.21.15.110:0/3609148265 shutdown_connections 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/3609148265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1eb006a8e0 0x7f1eb010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/3609148265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb01082b0 0x7f1eb0108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/3609148265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb0069f40 0x7f1eb006a3a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 -- 172.21.15.110:0/3609148265 >> 172.21.15.110:0/3609148265 conn(0x7f1eb00780e0 msgr2=0x7f1eb00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 -- 172.21.15.110:0/3609148265 shutdown_connections 2024-04-25T00:01:00.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.678+0000 7f1eb4b21700 1 -- 172.21.15.110:0/3609148265 wait complete. 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.679+0000 7f1eb4b21700 1 Processor -- start 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eb4b21700 1 -- start start 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eb4b21700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1eb0069f40 0x7f1eb0101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eb4b21700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb006a8e0 0x7f1eb0101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eb4b21700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb01082b0 0x7f1eb0101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eb4b21700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01124a0 con 0x7f1eb01082b0 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eb4b21700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0112320 con 0x7f1eb0069f40 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eb4b21700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0112620 con 0x7f1eb006a8e0 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eaed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb01082b0 0x7f1eb0101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eaed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb01082b0 0x7f1eb0101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49122/0 (socket says 172.21.15.110:49122) 2024-04-25T00:01:00.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eaed9d700 1 -- 172.21.15.110:0/1293963857 learned_addr learned my addr 172.21.15.110:0/1293963857 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1ea5d9b700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb006a8e0 0x7f1eb0101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eae59c700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1eb0069f40 0x7f1eb0101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.680+0000 7f1eaed9d700 1 -- 172.21.15.110:0/1293963857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb006a8e0 msgr2=0x7f1eb0101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1eaed9d700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb006a8e0 0x7f1eb0101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1eaed9d700 1 -- 172.21.15.110:0/1293963857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1eb0069f40 msgr2=0x7f1eb0101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1eaed9d700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1eb0069f40 0x7f1eb0101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1eaed9d700 1 -- 172.21.15.110:0/1293963857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb01a7020 con 0x7f1eb01082b0 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1ea5d9b700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb006a8e0 0x7f1eb0101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:00.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1eaed9d700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb01082b0 0x7f1eb0101c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800b970 tx=0x7f1ea800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.683 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:00.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1ea7fff700 1 -- 172.21.15.110:0/1293963857 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8019070 con 0x7f1eb01082b0 2024-04-25T00:01:00.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1ea7fff700 1 -- 172.21.15.110:0/1293963857 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ea800cde0 con 0x7f1eb01082b0 2024-04-25T00:01:00.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb01a7310 con 0x7f1eb01082b0 2024-04-25T00:01:00.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1ea7fff700 1 -- 172.21.15.110:0/1293963857 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8007370 con 0x7f1eb01082b0 2024-04-25T00:01:00.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.681+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01a7850 con 0x7f1eb01082b0 2024-04-25T00:01:00.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.682+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb010a820 con 0x7f1eb01082b0 2024-04-25T00:01:00.684 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:00.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.683+0000 7f1ea7fff700 1 -- 172.21.15.110:0/1293963857 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1ea801c070 con 0x7f1eb01082b0 2024-04-25T00:01:00.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.684+0000 7f1ea7fff700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1e9c0a2d30 0x7f1e9c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:00.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.684+0000 7f1eae59c700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1e9c0a2d30 0x7f1e9c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:00.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.684+0000 7f1ea7fff700 1 -- 172.21.15.110:0/1293963857 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1ea80ce580 con 0x7f1eb01082b0 2024-04-25T00:01:00.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.685+0000 7f1eae59c700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1e9c0a2d30 0x7f1e9c0a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009a10 tx=0x7f1ea0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:00.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.689+0000 7f1ea7fff700 1 -- 172.21.15.110:0/1293963857 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1ea809b460 con 0x7f1eb01082b0 2024-04-25T00:01:00.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.897+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1eb0102730 con 0x7f1eb01082b0 2024-04-25T00:01:00.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.897+0000 7f1ea7fff700 1 -- 172.21.15.110:0/1293963857 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1ea809ad60 con 0x7f1eb01082b0 2024-04-25T00:01:00.898 INFO:teuthology.orchestra.run.smithi110.stdout:163208757304 2024-04-25T00:01:00.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1e9c0a2d30 msgr2=0x7f1e9c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1e9c0a2d30 0x7f1e9c0a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009a10 tx=0x7f1ea0009190 comp rx=0 tx=0).stop 2024-04-25T00:01:00.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb01082b0 msgr2=0x7f1eb0101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:00.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb01082b0 0x7f1eb0101c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800b970 tx=0x7f1ea800bc80 comp rx=0 tx=0).stop 2024-04-25T00:01:00.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 shutdown_connections 2024-04-25T00:01:00.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1e9c0a2d30 0x7f1e9c0a51f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb01082b0 0x7f1eb0101c80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1eb0069f40 0x7f1eb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 --2- 172.21.15.110:0/1293963857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb006a8e0 0x7f1eb0101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:00.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.899+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 >> 172.21.15.110:0/1293963857 conn(0x7f1eb00780e0 msgr2=0x7f1eb00ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:00.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.900+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 shutdown_connections 2024-04-25T00:01:00.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:00.900+0000 7f1eb4b21700 1 -- 172.21.15.110:0/1293963857 wait complete. 2024-04-25T00:01:01.620 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757301 got 163208757304 for osd.5 2024-04-25T00:01:01.621 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 -- 172.21.15.110:0/2107476933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c102ef0 msgr2=0x7f416c106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 --2- 172.21.15.110:0/2107476933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c102ef0 0x7f416c106ec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4160009990 tx=0x7f4160009ca0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 -- 172.21.15.110:0/2107476933 shutdown_connections 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 --2- 172.21.15.110:0/2107476933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f416c1075a0 0x7f416c10f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 --2- 172.21.15.110:0/2107476933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f416c102540 0x7f416c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 --2- 172.21.15.110:0/2107476933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c102ef0 0x7f416c106ec0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 -- 172.21.15.110:0/2107476933 >> 172.21.15.110:0/2107476933 conn(0x7f416c0792c0 msgr2=0x7f416c0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 -- 172.21.15.110:0/2107476933 shutdown_connections 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.798+0000 7f4171998700 1 -- 172.21.15.110:0/2107476933 wait complete. 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f4171998700 1 Processor -- start 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f4171998700 1 -- start start 2024-04-25T00:01:01.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f4171998700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f416c102540 0x7f416c074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f4171998700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f416c102ef0 0x7f416c074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f4171998700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c1075a0 0x7f416c074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f4171998700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f416c1124f0 con 0x7f416c1075a0 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f4171998700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f416c112370 con 0x7f416c102540 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f4171998700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f416c112670 con 0x7f416c102ef0 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f416b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c1075a0 0x7f416c074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f416b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c1075a0 0x7f416c074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49152/0 (socket says 172.21.15.110:49152) 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f416b7fe700 1 -- 172.21.15.110:0/954413722 learned_addr learned my addr 172.21.15.110:0/954413722 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.799+0000 7f416b7fe700 1 -- 172.21.15.110:0/954413722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f416c102ef0 msgr2=0x7f416c074a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f416b7fe700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f416c102ef0 0x7f416c074a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f416b7fe700 1 -- 172.21.15.110:0/954413722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f416c102540 msgr2=0x7f416c074530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f416b7fe700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f416c102540 0x7f416c074530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:01.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f416b7fe700 1 -- 172.21.15.110:0/954413722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1a6f30 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f416b7fe700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c1075a0 0x7f416c074fb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f416400b970 tx=0x7f416400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f4170996700 1 -- 172.21.15.110:0/954413722 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164019070 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f4170996700 1 -- 172.21.15.110:0/954413722 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4164014440 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1a7220 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f4170996700 1 -- 172.21.15.110:0/954413722 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416400ce60 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.800+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a7710 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.801+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c068f20 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.802+0000 7f4170996700 1 -- 172.21.15.110:0/954413722 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f416401c070 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.803+0000 7f4170996700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f41540a2cb0 0x7f41540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.803+0000 7f416affd700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f41540a2cb0 0x7f41540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.803+0000 7f4170996700 1 -- 172.21.15.110:0/954413722 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f41640ce9e0 con 0x7f416c1075a0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.803+0000 7f416affd700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f41540a2cb0 0x7f41540a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f415c009730 tx=0x7f415c006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:01.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:01.806+0000 7f4170996700 1 -- 172.21.15.110:0/954413722 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f416409b920 con 0x7f416c1075a0 2024-04-25T00:01:01.912 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:01 smithi110 ceph-mon[29187]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:01.912 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:01 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1293963857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:01:02.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.012+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f416c02db30 con 0x7f416c1075a0 2024-04-25T00:01:02.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.012+0000 7f4170996700 1 -- 172.21.15.110:0/954413722 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f416409b220 con 0x7f416c1075a0 2024-04-25T00:01:02.013 INFO:teuthology.orchestra.run.smithi110.stdout:34359738449 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f41540a2cb0 msgr2=0x7f41540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f41540a2cb0 0x7f41540a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f415c009730 tx=0x7f415c006cb0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c1075a0 msgr2=0x7f416c074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c1075a0 0x7f416c074fb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f416400b970 tx=0x7f416400bd30 comp rx=0 tx=0).stop 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 shutdown_connections 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f41540a2cb0 0x7f41540a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c1075a0 0x7f416c074fb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f416c102540 0x7f416c074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 --2- 172.21.15.110:0/954413722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f416c102ef0 0x7f416c074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:02.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.014+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 >> 172.21.15.110:0/954413722 conn(0x7f416c0792c0 msgr2=0x7f416c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:02.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.015+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 shutdown_connections 2024-04-25T00:01:02.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:02.015+0000 7f4171998700 1 -- 172.21.15.110:0/954413722 wait complete. 2024-04-25T00:01:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:01 smithi188 ceph-mon[33285]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:01 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1293963857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:01:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:01 smithi173 ceph-mon[32402]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:01 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1293963857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:01:02.244 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:02.244 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:02.904 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:02 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/954413722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:01:02.904 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:02 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:01:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:02 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/954413722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:01:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:02 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:01:03.041 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:02 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/954413722' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:01:03.041 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:02 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:01:03.970 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:03 smithi110 ceph-mon[29187]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:03 smithi188 ceph-mon[33285]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:03 smithi173 ceph-mon[32402]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:05.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:04 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:04 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:04 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:04 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:04 smithi110 ceph-mon[29187]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:05.108 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738449 for osd.0 2024-04-25T00:01:05.108 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:05.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:04 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:04 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:04 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:04 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:04 smithi188 ceph-mon[33285]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:04 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:04 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:04 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:04 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:04 smithi173 ceph-mon[32402]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:06.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.020+0000 7fd9175f4700 1 -- 172.21.15.110:0/1724494455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910107420 msgr2=0x7fd910107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:06.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.020+0000 7fd9175f4700 1 --2- 172.21.15.110:0/1724494455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910107420 0x7fd910107800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0099c0 tx=0x7fd90c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.020+0000 7fd9175f4700 1 -- 172.21.15.110:0/1724494455 shutdown_connections 2024-04-25T00:01:06.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.020+0000 7fd9175f4700 1 --2- 172.21.15.110:0/1724494455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9101026c0 0x7fd91010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.020+0000 7fd9175f4700 1 --2- 172.21.15.110:0/1724494455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910107420 0x7fd910107800 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.020+0000 7fd9175f4700 1 --2- 172.21.15.110:0/1724494455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd910101d00 0x7fd910102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.020+0000 7fd9175f4700 1 -- 172.21.15.110:0/1724494455 >> 172.21.15.110:0/1724494455 conn(0x7fd910076450 msgr2=0x7fd910076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:06.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.021+0000 7fd9175f4700 1 -- 172.21.15.110:0/1724494455 shutdown_connections 2024-04-25T00:01:06.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.021+0000 7fd9175f4700 1 -- 172.21.15.110:0/1724494455 wait complete. 2024-04-25T00:01:06.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.021+0000 7fd9175f4700 1 Processor -- start 2024-04-25T00:01:06.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.021+0000 7fd9175f4700 1 -- start start 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9175f4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910101d00 0x7fd910199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9175f4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9101026c0 0x7fd910199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9175f4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd910107420 0x7fd9101a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9175f4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101114c0 con 0x7fd910101d00 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9175f4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd910111340 con 0x7fd910107420 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9175f4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd910111640 con 0x7fd9101026c0 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910101d00 0x7fd910199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910101d00 0x7fd910199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49176/0 (socket says 172.21.15.110:49176) 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 -- 172.21.15.110:0/3828131381 learned_addr learned my addr 172.21.15.110:0/3828131381 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915b91700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd910107420 0x7fd9101a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 -- 172.21.15.110:0/3828131381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9101026c0 msgr2=0x7fd910199da0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9101026c0 0x7fd910199da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 -- 172.21.15.110:0/3828131381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd910107420 msgr2=0x7fd9101a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd910107420 0x7fd9101a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 -- 172.21.15.110:0/3828131381 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9101a4e20 con 0x7fd910101d00 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915b91700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd910107420 0x7fd9101a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd915390700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910101d00 0x7fd910199860 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd90c006010 tx=0x7fd90c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9067fc700 1 -- 172.21.15.110:0/3828131381 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c027070 con 0x7fd910101d00 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9067fc700 1 -- 172.21.15.110:0/3828131381 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd90c003fd0 con 0x7fd910101d00 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9101a4ff0 con 0x7fd910101d00 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9067fc700 1 -- 172.21.15.110:0/3828131381 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c022d70 con 0x7fd910101d00 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.022+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9101a53e0 con 0x7fd910101d00 2024-04-25T00:01:06.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.023+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd910068f20 con 0x7fd910101d00 2024-04-25T00:01:06.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.025+0000 7fd9067fc700 1 -- 172.21.15.110:0/3828131381 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd90c02a070 con 0x7fd910101d00 2024-04-25T00:01:06.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.025+0000 7fd9067fc700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd8fc0a2cb0 0x7fd8fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:06.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.026+0000 7fd914b8f700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd8fc0a2cb0 0x7fd8fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:06.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.026+0000 7fd9067fc700 1 -- 172.21.15.110:0/3828131381 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd90c0d0080 con 0x7fd910101d00 2024-04-25T00:01:06.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.026+0000 7fd914b8f700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd8fc0a2cb0 0x7fd8fc0a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9100769c0 tx=0x7fd90000b3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:06.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.029+0000 7fd9067fc700 1 -- 172.21.15.110:0/3828131381 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd90c013070 con 0x7fd910101d00 2024-04-25T00:01:06.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.233+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd9101a5820 con 0x7fd910101d00 2024-04-25T00:01:06.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.233+0000 7fd9067fc700 1 -- 172.21.15.110:0/3828131381 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd90c09cf40 con 0x7fd910101d00 2024-04-25T00:01:06.234 INFO:teuthology.orchestra.run.smithi110.stdout:107374182467 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd8fc0a2cb0 msgr2=0x7fd8fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd8fc0a2cb0 0x7fd8fc0a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9100769c0 tx=0x7fd90000b3f0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910101d00 msgr2=0x7fd910199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910101d00 0x7fd910199860 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd90c006010 tx=0x7fd90c009f80 comp rx=0 tx=0).stop 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 shutdown_connections 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd8fc0a2cb0 0x7fd8fc0a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd910107420 0x7fd9101a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd910101d00 0x7fd910199860 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 --2- 172.21.15.110:0/3828131381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9101026c0 0x7fd910199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.235+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 >> 172.21.15.110:0/3828131381 conn(0x7fd910076450 msgr2=0x7fd9100fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:06.236 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.236+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 shutdown_connections 2024-04-25T00:01:06.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:06.236+0000 7fd9175f4700 1 -- 172.21.15.110:0/3828131381 wait complete. 2024-04-25T00:01:06.522 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:06 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3828131381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:01:06.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:06 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3828131381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:01:06.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:06 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3828131381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:01:07.526 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:07 smithi110 ceph-mon[29187]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:07.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:07 smithi173 ceph-mon[32402]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:07 smithi188 ceph-mon[33285]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:08.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 -- 172.21.15.110:0/385006536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 msgr2=0x7f426410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 --2- 172.21.15.110:0/385006536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 0x7f426410dfe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f425c00b0a0 tx=0x7f425c00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 -- 172.21.15.110:0/385006536 shutdown_connections 2024-04-25T00:01:08.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 --2- 172.21.15.110:0/385006536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 0x7f426410dfe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 --2- 172.21.15.110:0/385006536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42641071a0 0x7f4264107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 --2- 172.21.15.110:0/385006536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4264101130 0x7f42641015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 -- 172.21.15.110:0/385006536 >> 172.21.15.110:0/385006536 conn(0x7f42640780e0 msgr2=0x7f42640784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 -- 172.21.15.110:0/385006536 shutdown_connections 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 -- 172.21.15.110:0/385006536 wait complete. 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 Processor -- start 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.600+0000 7f426c675700 1 -- start start 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426c675700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4264101130 0x7f4264199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426c675700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 0x7f4264199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426c675700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42641071a0 0x7f42641a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426c675700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4264111330 con 0x7f4264101af0 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426c675700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f42641111b0 con 0x7f4264101130 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426c675700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42641114b0 con 0x7f42641071a0 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 0x7f4264199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 0x7f4264199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38626/0 (socket says 172.21.15.110:38626) 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 -- 172.21.15.110:0/2464971339 learned_addr learned my addr 172.21.15.110:0/2464971339 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426a411700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4264101130 0x7f4264199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426ac12700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42641071a0 0x7f42641a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 -- 172.21.15.110:0/2464971339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42641071a0 msgr2=0x7f42641a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42641071a0 0x7f42641a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 -- 172.21.15.110:0/2464971339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4264101130 msgr2=0x7f4264199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4264101130 0x7f4264199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 -- 172.21.15.110:0/2464971339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42641a4d50 con 0x7f4264101af0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f4269c10700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 0x7f4264199cd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f425400d980 tx=0x7f425400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f425b7fe700 1 -- 172.21.15.110:0/2464971339 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254016070 con 0x7f4264101af0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f425b7fe700 1 -- 172.21.15.110:0/2464971339 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254004830 con 0x7f4264101af0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42641a4f80 con 0x7f4264101af0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f425b7fe700 1 -- 172.21.15.110:0/2464971339 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254005530 con 0x7f4264101af0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.601+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42641a5430 con 0x7f4264101af0 2024-04-25T00:01:08.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.602+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4264102850 con 0x7f4264101af0 2024-04-25T00:01:08.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.603+0000 7f425b7fe700 1 -- 172.21.15.110:0/2464971339 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f42540049e0 con 0x7f4264101af0 2024-04-25T00:01:08.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.604+0000 7f425b7fe700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f42500a2cb0 0x7f42500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:08.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.604+0000 7f426a411700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f42500a2cb0 0x7f42500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:08.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.604+0000 7f425b7fe700 1 -- 172.21.15.110:0/2464971339 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f42540cf3c0 con 0x7f4264101af0 2024-04-25T00:01:08.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.604+0000 7f426a411700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f42500a2cb0 0x7f42500a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4260005950 tx=0x7f426000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:08.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.608+0000 7f425b7fe700 1 -- 172.21.15.110:0/2464971339 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f425401f020 con 0x7f4264101af0 2024-04-25T00:01:08.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.814+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f426402db30 con 0x7f4264101af0 2024-04-25T00:01:08.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.814+0000 7f425b7fe700 1 -- 172.21.15.110:0/2464971339 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f425409c1f0 con 0x7f4264101af0 2024-04-25T00:01:08.814 INFO:teuthology.orchestra.run.smithi110.stdout:197568495668 2024-04-25T00:01:08.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f42500a2cb0 msgr2=0x7f42500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f42500a2cb0 0x7f42500a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4260005950 tx=0x7f426000a300 comp rx=0 tx=0).stop 2024-04-25T00:01:08.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 msgr2=0x7f4264199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:08.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 0x7f4264199cd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f425400d980 tx=0x7f425400dc90 comp rx=0 tx=0).stop 2024-04-25T00:01:08.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 shutdown_connections 2024-04-25T00:01:08.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f42500a2cb0 0x7f42500a5170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42641071a0 0x7f42641a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4264101130 0x7f4264199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 --2- 172.21.15.110:0/2464971339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4264101af0 0x7f4264199cd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:08.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 >> 172.21.15.110:0/2464971339 conn(0x7f42640780e0 msgr2=0x7f42640ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:08.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 shutdown_connections 2024-04-25T00:01:08.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:08.816+0000 7f426c675700 1 -- 172.21.15.110:0/2464971339 wait complete. 2024-04-25T00:01:09.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:09 smithi110 ceph-mon[29187]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:09.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:09 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2464971339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:01:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:09 smithi173 ceph-mon[32402]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:09.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:09 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2464971339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:01:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:09 smithi188 ceph-mon[33285]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:09 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2464971339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 -- 172.21.15.110:0/3302459640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102ec0 msgr2=0x7f3598106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 --2- 172.21.15.110:0/3302459640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102ec0 0x7f3598106f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f358c0099c0 tx=0x7f358c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 -- 172.21.15.110:0/3302459640 shutdown_connections 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 --2- 172.21.15.110:0/3302459640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981075e0 0x7f359810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 --2- 172.21.15.110:0/3302459640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598102510 0x7f35981028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 --2- 172.21.15.110:0/3302459640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102ec0 0x7f3598106f00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 -- 172.21.15.110:0/3302459640 >> 172.21.15.110:0/3302459640 conn(0x7f35980793e0 msgr2=0x7f35980797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 -- 172.21.15.110:0/3302459640 shutdown_connections 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.591+0000 7f359e728700 1 -- 172.21.15.110:0/3302459640 wait complete. 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f359e728700 1 Processor -- start 2024-04-25T00:01:10.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f359e728700 1 -- start start 2024-04-25T00:01:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f359e728700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102510 0x7f3598074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f359e728700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598102ec0 0x7f3598074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f359e728700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981075e0 0x7f3598074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f359e728700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3598112550 con 0x7f3598102510 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f359e728700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35981123d0 con 0x7f35981075e0 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f359e728700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35981126d0 con 0x7f3598102ec0 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f3597fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102510 0x7f3598074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f3597fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102510 0x7f3598074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38646/0 (socket says 172.21.15.110:38646) 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.592+0000 7f3597fff700 1 -- 172.21.15.110:0/2686565568 learned_addr learned my addr 172.21.15.110:0/2686565568 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f35977fe700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598102ec0 0x7f3598074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f359ccc5700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981075e0 0x7f3598074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f3597fff700 1 -- 172.21.15.110:0/2686565568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598102ec0 msgr2=0x7f3598074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f3597fff700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598102ec0 0x7f3598074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f3597fff700 1 -- 172.21.15.110:0/2686565568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981075e0 msgr2=0x7f3598074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f3597fff700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981075e0 0x7f3598074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f3597fff700 1 -- 172.21.15.110:0/2686565568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35981a6fa0 con 0x7f3598102510 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f359ccc5700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981075e0 0x7f3598074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:10.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f3597fff700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102510 0x7f3598074530 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f358800b530 tx=0x7f358800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f35957fa700 1 -- 172.21.15.110:0/2686565568 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588015020 con 0x7f3598102510 2024-04-25T00:01:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f35957fa700 1 -- 172.21.15.110:0/2686565568 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3588011ce0 con 0x7f3598102510 2024-04-25T00:01:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35981a7290 con 0x7f3598102510 2024-04-25T00:01:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f35957fa700 1 -- 172.21.15.110:0/2686565568 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588010e40 con 0x7f3598102510 2024-04-25T00:01:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.593+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35981a7750 con 0x7f3598102510 2024-04-25T00:01:10.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.594+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35981039e0 con 0x7f3598102510 2024-04-25T00:01:10.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.596+0000 7f35957fa700 1 -- 172.21.15.110:0/2686565568 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3588018020 con 0x7f3598102510 2024-04-25T00:01:10.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.596+0000 7f35957fa700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2cb0 0x7f35800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:10.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.597+0000 7f35977fe700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2cb0 0x7f35800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:10.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.597+0000 7f35957fa700 1 -- 172.21.15.110:0/2686565568 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f35880ceec0 con 0x7f3598102510 2024-04-25T00:01:10.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.597+0000 7f35977fe700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2cb0 0x7f35800a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f358c009990 tx=0x7f358c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:10.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.600+0000 7f35957fa700 1 -- 172.21.15.110:0/2686565568 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f358801f020 con 0x7f3598102510 2024-04-25T00:01:10.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.810+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f359802db30 con 0x7f3598102510 2024-04-25T00:01:10.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.811+0000 7f35957fa700 1 -- 172.21.15.110:0/2686565568 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3588010380 con 0x7f3598102510 2024-04-25T00:01:10.811 INFO:teuthology.orchestra.run.smithi110.stdout:309237645344 2024-04-25T00:01:10.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2cb0 msgr2=0x7f35800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2cb0 0x7f35800a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f358c009990 tx=0x7f358c005fb0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102510 msgr2=0x7f3598074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:10.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102510 0x7f3598074530 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f358800b530 tx=0x7f358800b840 comp rx=0 tx=0).stop 2024-04-25T00:01:10.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 shutdown_connections 2024-04-25T00:01:10.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2cb0 0x7f35800a5170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981075e0 0x7f3598074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598102510 0x7f3598074530 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 --2- 172.21.15.110:0/2686565568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598102ec0 0x7f3598074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:10.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 >> 172.21.15.110:0/2686565568 conn(0x7f35980793e0 msgr2=0x7f35981003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:10.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 shutdown_connections 2024-04-25T00:01:10.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:10.813+0000 7f359e728700 1 -- 172.21.15.110:0/2686565568 wait complete. 2024-04-25T00:01:11.654 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:11 smithi110 ceph-mon[29187]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:11.654 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:11 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2686565568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:01:11.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:11 smithi173 ceph-mon[32402]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:11.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:11 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2686565568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:01:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:11 smithi188 ceph-mon[33285]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:01:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:11 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2686565568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:01:12.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 -- 172.21.15.110:0/1782604175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc069f40 msgr2=0x7f24cc06a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 --2- 172.21.15.110:0/1782604175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc069f40 0x7f24cc06a3a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009990 tx=0x7f24b8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 -- 172.21.15.110:0/1782604175 shutdown_connections 2024-04-25T00:01:12.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 --2- 172.21.15.110:0/1782604175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24cc06a8e0 0x7f24cc10f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 --2- 172.21.15.110:0/1782604175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24cc1082b0 0x7f24cc108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 --2- 172.21.15.110:0/1782604175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc069f40 0x7f24cc06a3a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 -- 172.21.15.110:0/1782604175 >> 172.21.15.110:0/1782604175 conn(0x7f24cc0780e0 msgr2=0x7f24cc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 -- 172.21.15.110:0/1782604175 shutdown_connections 2024-04-25T00:01:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 -- 172.21.15.110:0/1782604175 wait complete. 2024-04-25T00:01:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.084+0000 7f24d3077700 1 Processor -- start 2024-04-25T00:01:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d3077700 1 -- start start 2024-04-25T00:01:12.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d3077700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24cc069f40 0x7f24cc19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d3077700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc06a8e0 0x7f24cc19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d3077700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24cc1082b0 0x7f24cc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d3077700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1123c0 con 0x7f24cc06a8e0 2024-04-25T00:01:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d3077700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc112240 con 0x7f24cc1082b0 2024-04-25T00:01:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d3077700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc112540 con 0x7f24cc069f40 2024-04-25T00:01:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc06a8e0 0x7f24cc19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc06a8e0 0x7f24cc19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38678/0 (socket says 172.21.15.110:38678) 2024-04-25T00:01:12.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 -- 172.21.15.110:0/3571279408 learned_addr learned my addr 172.21.15.110:0/3571279408 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d0e13700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24cc069f40 0x7f24cc19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 -- 172.21.15.110:0/3571279408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24cc069f40 msgr2=0x7f24cc19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24cc069f40 0x7f24cc19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d1614700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24cc1082b0 0x7f24cc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 -- 172.21.15.110:0/3571279408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24cc1082b0 msgr2=0x7f24cc198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24cc1082b0 0x7f24cc198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 -- 172.21.15.110:0/3571279408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc199000 con 0x7f24cc06a8e0 2024-04-25T00:01:12.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24d0e13700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24cc069f40 0x7f24cc19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:12.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.085+0000 7f24cbfff700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc06a8e0 0x7f24cc19eb90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f24b8003ab0 tx=0x7f24b8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.086+0000 7f24c9ffb700 1 -- 172.21.15.110:0/3571279408 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8027070 con 0x7f24cc06a8e0 2024-04-25T00:01:12.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.086+0000 7f24c9ffb700 1 -- 172.21.15.110:0/3571279408 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24b800bdc0 con 0x7f24cc06a8e0 2024-04-25T00:01:12.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.086+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc199290 con 0x7f24cc06a8e0 2024-04-25T00:01:12.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.086+0000 7f24c9ffb700 1 -- 172.21.15.110:0/3571279408 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8022d40 con 0x7f24cc06a8e0 2024-04-25T00:01:12.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.086+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc1a5400 con 0x7f24cc06a8e0 2024-04-25T00:01:12.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.087+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24cc103960 con 0x7f24cc06a8e0 2024-04-25T00:01:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.088+0000 7f24c9ffb700 1 -- 172.21.15.110:0/3571279408 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f24b802a070 con 0x7f24cc06a8e0 2024-04-25T00:01:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.088+0000 7f24c9ffb700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f24bc0a2cb0 0x7f24bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.089+0000 7f24d0e13700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f24bc0a2cb0 0x7f24bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.089+0000 7f24c9ffb700 1 -- 172.21.15.110:0/3571279408 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f24b80cfef0 con 0x7f24cc06a8e0 2024-04-25T00:01:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.089+0000 7f24d0e13700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f24bc0a2cb0 0x7f24bc0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f24c000bb90 tx=0x7f24c0005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.093+0000 7f24c9ffb700 1 -- 172.21.15.110:0/3571279408 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f24b805e740 con 0x7f24cc06a8e0 2024-04-25T00:01:12.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.291+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f24cc02db30 con 0x7f24cc06a8e0 2024-04-25T00:01:12.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.291+0000 7f24c9ffb700 1 -- 172.21.15.110:0/3571279408 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f24b8013070 con 0x7f24cc06a8e0 2024-04-25T00:01:12.291 INFO:teuthology.orchestra.run.smithi110.stdout:141733920831 2024-04-25T00:01:12.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f24bc0a2cb0 msgr2=0x7f24bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f24bc0a2cb0 0x7f24bc0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f24c000bb90 tx=0x7f24c0005d80 comp rx=0 tx=0).stop 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc06a8e0 msgr2=0x7f24cc19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc06a8e0 0x7f24cc19eb90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f24b8003ab0 tx=0x7f24b8003b90 comp rx=0 tx=0).stop 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 shutdown_connections 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f24bc0a2cb0 0x7f24bc0a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24cc1082b0 0x7f24cc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24cc069f40 0x7f24cc19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 --2- 172.21.15.110:0/3571279408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24cc06a8e0 0x7f24cc19eb90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 >> 172.21.15.110:0/3571279408 conn(0x7f24cc0780e0 msgr2=0x7f24cc0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 shutdown_connections 2024-04-25T00:01:12.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:12.293+0000 7f24d3077700 1 -- 172.21.15.110:0/3571279408 wait complete. 2024-04-25T00:01:12.609 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645339 got 309237645344 for osd.10 2024-04-25T00:01:12.609 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:12.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 -- 172.21.15.110:0/3906056663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c1075a0 msgr2=0x7fd21c10f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 --2- 172.21.15.110:0/3906056663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c1075a0 0x7fd21c10f170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd21400b0a0 tx=0x7fd21400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 -- 172.21.15.110:0/3906056663 shutdown_connections 2024-04-25T00:01:12.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 --2- 172.21.15.110:0/3906056663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c1075a0 0x7fd21c10f170 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 --2- 172.21.15.110:0/3906056663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c1024e0 0x7fd21c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 --2- 172.21.15.110:0/3906056663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd21c102e90 0x7fd21c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 -- 172.21.15.110:0/3906056663 >> 172.21.15.110:0/3906056663 conn(0x7fd21c0793e0 msgr2=0x7fd21c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 -- 172.21.15.110:0/3906056663 shutdown_connections 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.752+0000 7fd2247ef700 1 -- 172.21.15.110:0/3906056663 wait complete. 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd2247ef700 1 Processor -- start 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd2247ef700 1 -- start start 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd2247ef700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c1024e0 0x7fd21c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd2247ef700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c102e90 0x7fd21c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd2247ef700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd21c1075a0 0x7fd21c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd2247ef700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1124b0 con 0x7fd21c102e90 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd2247ef700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c112330 con 0x7fd21c1075a0 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd2247ef700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c112630 con 0x7fd21c1024e0 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd221d8a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c102e90 0x7fd21c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd221d8a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c102e90 0x7fd21c19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38658/0 (socket says 172.21.15.110:38658) 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd221d8a700 1 -- 172.21.15.110:0/2343667783 learned_addr learned my addr 172.21.15.110:0/2343667783 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd22258b700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c1024e0 0x7fd21c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd221d8a700 1 -- 172.21.15.110:0/2343667783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c1024e0 msgr2=0x7fd21c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd221d8a700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c1024e0 0x7fd21c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd221d8a700 1 -- 172.21.15.110:0/2343667783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd21c1075a0 msgr2=0x7fd21c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd221d8a700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd21c1075a0 0x7fd21c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.753+0000 7fd221d8a700 1 -- 172.21.15.110:0/2343667783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c198f70 con 0x7fd21c102e90 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.754+0000 7fd221d8a700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c102e90 0x7fd21c19eb50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b500 tx=0x7fd20c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.754+0000 7fd2137fe700 1 -- 172.21.15.110:0/2343667783 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c015020 con 0x7fd21c102e90 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.754+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c199260 con 0x7fd21c102e90 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.754+0000 7fd2137fe700 1 -- 172.21.15.110:0/2343667783 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd20c011ce0 con 0x7fd21c102e90 2024-04-25T00:01:12.685 INFO:teuthology.orchestra.run.smithi110.stdout:343597383706 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.754+0000 7fd2137fe700 1 -- 172.21.15.110:0/2343667783 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c010d40 con 0x7fd21c102e90 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.754+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1a5470 con 0x7fd21c102e90 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.755+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21c1039a0 con 0x7fd21c102e90 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.756+0000 7fd2137fe700 1 -- 172.21.15.110:0/2343667783 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd20c018020 con 0x7fd21c102e90 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.756+0000 7fd2137fe700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd2080a2cb0 0x7fd2080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.756+0000 7fd22258b700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd2080a2cb0 0x7fd2080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.756+0000 7fd2137fe700 1 -- 172.21.15.110:0/2343667783 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd20c0ced50 con 0x7fd21c102e90 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.757+0000 7fd22258b700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd2080a2cb0 0x7fd2080a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd218004630 tx=0x7fd21800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.760+0000 7fd2137fe700 1 -- 172.21.15.110:0/2343667783 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd20c01f020 con 0x7fd21c102e90 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.959+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd21c02db30 con 0x7fd21c102e90 2024-04-25T00:01:12.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.959+0000 7fd2137fe700 1 -- 172.21.15.110:0/2343667783 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd20c09bc90 con 0x7fd21c102e90 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd2080a2cb0 msgr2=0x7fd2080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd2080a2cb0 0x7fd2080a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd218004630 tx=0x7fd21800b3c0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c102e90 msgr2=0x7fd21c19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c102e90 0x7fd21c19eb50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b500 tx=0x7fd20c00b810 comp rx=0 tx=0).stop 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 shutdown_connections 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd2080a2cb0 0x7fd2080a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd21c1075a0 0x7fd21c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c1024e0 0x7fd21c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 --2- 172.21.15.110:0/2343667783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd21c102e90 0x7fd21c19eb50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 >> 172.21.15.110:0/2343667783 conn(0x7fd21c0793e0 msgr2=0x7fd21c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 shutdown_connections 2024-04-25T00:01:12.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:11.961+0000 7fd2247ef700 1 -- 172.21.15.110:0/2343667783 wait complete. 2024-04-25T00:01:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:12 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2343667783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:01:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:12 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3571279408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:01:12.934 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:12 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2343667783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:01:12.934 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:12 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3571279408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:01:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:12 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2343667783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:01:13.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:12 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3571279408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:01:13.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182467 for osd.3 2024-04-25T00:01:13.653 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:13.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:13 smithi173 ceph-mon[32402]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:13.984 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:13 smithi110 ceph-mon[29187]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:13 smithi188 ceph-mon[33285]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:14.663 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920824 got 141733920831 for osd.4 2024-04-25T00:01:14.663 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:15.371 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495668 for osd.6 2024-04-25T00:01:15.371 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:15.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 -- 172.21.15.110:0/2763003916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102ec0 msgr2=0x7f76c8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 --2- 172.21.15.110:0/2763003916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102ec0 0x7f76c8106f00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0099c0 tx=0x7f76bc009cd0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 -- 172.21.15.110:0/2763003916 shutdown_connections 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 --2- 172.21.15.110:0/2763003916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76c81075e0 0x7f76c810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 --2- 172.21.15.110:0/2763003916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76c8102510 0x7f76c81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 --2- 172.21.15.110:0/2763003916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102ec0 0x7f76c8106f00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 -- 172.21.15.110:0/2763003916 >> 172.21.15.110:0/2763003916 conn(0x7f76c80793e0 msgr2=0x7f76c80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 -- 172.21.15.110:0/2763003916 shutdown_connections 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.329+0000 7f76ccc42700 1 -- 172.21.15.110:0/2763003916 wait complete. 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76ccc42700 1 Processor -- start 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76ccc42700 1 -- start start 2024-04-25T00:01:15.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76ccc42700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102510 0x7f76c8074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76ccc42700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76c8102ec0 0x7f76c8074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76ccc42700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76c81075e0 0x7f76c8074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76ccc42700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8112480 con 0x7f76c8102510 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76ccc42700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8112300 con 0x7f76c81075e0 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76ccc42700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8112600 con 0x7f76c8102ec0 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102510 0x7f76c8074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102510 0x7f76c8074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38692/0 (socket says 172.21.15.110:38692) 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 -- 172.21.15.110:0/1477519330 learned_addr learned my addr 172.21.15.110:0/1477519330 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c6d9d700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76c81075e0 0x7f76c8074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c5d9b700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76c8102ec0 0x7f76c8074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 -- 172.21.15.110:0/1477519330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76c8102ec0 msgr2=0x7f76c8074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76c8102ec0 0x7f76c8074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 -- 172.21.15.110:0/1477519330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76c81075e0 msgr2=0x7f76c8074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:15.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76c81075e0 0x7f76c8074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 -- 172.21.15.110:0/1477519330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c81a6ee0 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.330+0000 7f76c659c700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102510 0x7f76c8074530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f76b800c920 tx=0x7f76b800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.331+0000 7f76b77fe700 1 -- 172.21.15.110:0/1477519330 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8007b00 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.331+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c81a71d0 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.331+0000 7f76b77fe700 1 -- 172.21.15.110:0/1477519330 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76b8004ce0 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.331+0000 7f76b77fe700 1 -- 172.21.15.110:0/1477519330 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8005560 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.331+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81a7730 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.332+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c81039e0 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.334+0000 7f76b77fe700 1 -- 172.21.15.110:0/1477519330 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f76b8004750 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.334+0000 7f76b77fe700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76b00a2cb0 0x7f76b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.334+0000 7f76c5d9b700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76b00a2cb0 0x7f76b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.334+0000 7f76b77fe700 1 -- 172.21.15.110:0/1477519330 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f76b80cf1d0 con 0x7f76c8102510 2024-04-25T00:01:15.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.334+0000 7f76c5d9b700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76b00a2cb0 0x7f76b00a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000c00 tx=0x7f76bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:15.463 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.338+0000 7f76b77fe700 1 -- 172.21.15.110:0/1477519330 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f76b8014030 con 0x7f76c8102510 2024-04-25T00:01:15.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.546+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f76c8075a60 con 0x7f76c8102510 2024-04-25T00:01:15.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.546+0000 7f76b77fe700 1 -- 172.21.15.110:0/1477519330 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f76b809c000 con 0x7f76c8102510 2024-04-25T00:01:15.546 INFO:teuthology.orchestra.run.smithi110.stdout:219043332144 2024-04-25T00:01:15.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76b00a2cb0 msgr2=0x7f76b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76b00a2cb0 0x7f76b00a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000c00 tx=0x7f76bc005fd0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102510 msgr2=0x7f76c8074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102510 0x7f76c8074530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f76b800c920 tx=0x7f76b800cce0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 shutdown_connections 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76b00a2cb0 0x7f76b00a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76c81075e0 0x7f76c8074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8102510 0x7f76c8074530 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 --2- 172.21.15.110:0/1477519330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76c8102ec0 0x7f76c8074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 >> 172.21.15.110:0/1477519330 conn(0x7f76c80793e0 msgr2=0x7f76c8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 shutdown_connections 2024-04-25T00:01:15.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:15.548+0000 7f76ccc42700 1 -- 172.21.15.110:0/1477519330 wait complete. 2024-04-25T00:01:15.689 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:15 smithi110 ceph-mon[29187]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:15.689 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:15 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1477519330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:01:15.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:15 smithi173 ceph-mon[32402]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:15.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:15 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1477519330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:01:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:15 smithi188 ceph-mon[33285]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:16.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:15 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1477519330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 -- 172.21.15.110:0/488271399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc1082c0 msgr2=0x7ff9bc1086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 --2- 172.21.15.110:0/488271399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc1082c0 0x7ff9bc1086a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009990 tx=0x7ff9b0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 -- 172.21.15.110:0/488271399 shutdown_connections 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 --2- 172.21.15.110:0/488271399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9bc1011c0 0x7ff9bc10f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 --2- 172.21.15.110:0/488271399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc1082c0 0x7ff9bc1086a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 --2- 172.21.15.110:0/488271399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9bc100820 0x7ff9bc100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 -- 172.21.15.110:0/488271399 >> 172.21.15.110:0/488271399 conn(0x7ff9bc0777a0 msgr2=0x7ff9bc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 -- 172.21.15.110:0/488271399 shutdown_connections 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 -- 172.21.15.110:0/488271399 wait complete. 2024-04-25T00:01:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 Processor -- start 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.361+0000 7ff9c325a700 1 -- start start 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c325a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc100820 0x7ff9bc19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c325a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9bc1011c0 0x7ff9bc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c325a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9bc1082c0 0x7ff9bc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c325a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc1123b0 con 0x7ff9bc100820 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c325a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc112230 con 0x7ff9bc1082c0 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c325a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc112530 con 0x7ff9bc1011c0 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc100820 0x7ff9bc19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc100820 0x7ff9bc19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38702/0 (socket says 172.21.15.110:38702) 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 -- 172.21.15.110:0/4159988161 learned_addr learned my addr 172.21.15.110:0/4159988161 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9bbfff700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9bc1011c0 0x7ff9bc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c17f7700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9bc1082c0 0x7ff9bc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 -- 172.21.15.110:0/4159988161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9bc1011c0 msgr2=0x7ff9bc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9bc1011c0 0x7ff9bc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 -- 172.21.15.110:0/4159988161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9bc1082c0 msgr2=0x7ff9bc1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9bc1082c0 0x7ff9bc1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 -- 172.21.15.110:0/4159988161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc198fd0 con 0x7ff9bc100820 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9bbfff700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9bc1011c0 0x7ff9bc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c0ff6700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc100820 0x7ff9bc19e620 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00057c0 tx=0x7ff9b000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9b9ffb700 1 -- 172.21.15.110:0/4159988161 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0026070 con 0x7ff9bc100820 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9bc199260 con 0x7ff9bc100820 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9b9ffb700 1 -- 172.21.15.110:0/4159988161 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9b000fbb0 con 0x7ff9bc100820 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.362+0000 7ff9b9ffb700 1 -- 172.21.15.110:0/4159988161 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00173f0 con 0x7ff9bc100820 2024-04-25T00:01:16.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.363+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9bc1a55a0 con 0x7ff9bc100820 2024-04-25T00:01:16.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.364+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9bc103b40 con 0x7ff9bc100820 2024-04-25T00:01:16.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.365+0000 7ff9b9ffb700 1 -- 172.21.15.110:0/4159988161 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff9b0029070 con 0x7ff9bc100820 2024-04-25T00:01:16.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.365+0000 7ff9b9ffb700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff9ac0a2cb0 0x7ff9ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:16.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.365+0000 7ff9bbfff700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff9ac0a2cb0 0x7ff9ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:16.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.365+0000 7ff9b9ffb700 1 -- 172.21.15.110:0/4159988161 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff9b00cfeb0 con 0x7ff9bc100820 2024-04-25T00:01:16.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.366+0000 7ff9bbfff700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff9ac0a2cb0 0x7ff9ac0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc199eb0 tx=0x7ff9a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:16.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.369+0000 7ff9b9ffb700 1 -- 172.21.15.110:0/4159988161 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff9b0013070 con 0x7ff9bc100820 2024-04-25T00:01:16.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.570+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff9bc068f50 con 0x7ff9bc100820 2024-04-25T00:01:16.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.571+0000 7ff9b9ffb700 1 -- 172.21.15.110:0/4159988161 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff9bc068f50 con 0x7ff9bc100820 2024-04-25T00:01:16.571 INFO:teuthology.orchestra.run.smithi110.stdout:287762808868 2024-04-25T00:01:16.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff9ac0a2cb0 msgr2=0x7ff9ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff9ac0a2cb0 0x7ff9ac0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc199eb0 tx=0x7ff9a8009260 comp rx=0 tx=0).stop 2024-04-25T00:01:16.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc100820 msgr2=0x7ff9bc19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc100820 0x7ff9bc19e620 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00057c0 tx=0x7ff9b000b730 comp rx=0 tx=0).stop 2024-04-25T00:01:16.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 shutdown_connections 2024-04-25T00:01:16.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff9ac0a2cb0 0x7ff9ac0a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9bc1082c0 0x7ff9bc1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9bc100820 0x7ff9bc19e620 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 --2- 172.21.15.110:0/4159988161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9bc1011c0 0x7ff9bc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 >> 172.21.15.110:0/4159988161 conn(0x7ff9bc0777a0 msgr2=0x7ff9bc0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:16.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 shutdown_connections 2024-04-25T00:01:16.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.573+0000 7ff9c325a700 1 -- 172.21.15.110:0/4159988161 wait complete. 2024-04-25T00:01:16.619 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:16 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4159988161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:01:16.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 -- 172.21.15.110:0/1274337147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4100101af0 msgr2=0x7f410010dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 --2- 172.21.15.110:0/1274337147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4100101af0 0x7f410010dfe0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f40f800b0a0 tx=0x7f40f800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 -- 172.21.15.110:0/1274337147 shutdown_connections 2024-04-25T00:01:16.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 --2- 172.21.15.110:0/1274337147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4100101af0 0x7f410010dfe0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 --2- 172.21.15.110:0/1274337147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41001071a0 0x7f4100107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 --2- 172.21.15.110:0/1274337147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4100101130 0x7f41001015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 -- 172.21.15.110:0/1274337147 >> 172.21.15.110:0/1274337147 conn(0x7f41000780e0 msgr2=0x7f41000784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:16.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 -- 172.21.15.110:0/1274337147 shutdown_connections 2024-04-25T00:01:16.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.620+0000 7f41065a8700 1 -- 172.21.15.110:0/1274337147 wait complete. 2024-04-25T00:01:16.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f41065a8700 1 Processor -- start 2024-04-25T00:01:16.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f41065a8700 1 -- start start 2024-04-25T00:01:16.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f41065a8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4100101130 0x7f410019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:16.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f41065a8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4100101af0 0x7f410019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:16.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f41065a8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41001071a0 0x7f41001986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:16.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f41065a8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4100111340 con 0x7f41001071a0 2024-04-25T00:01:16.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f41065a8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f41001111c0 con 0x7f4100101130 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f41065a8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41001114c0 con 0x7f4100101af0 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f4104b45700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41001071a0 0x7f41001986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f4104b45700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41001071a0 0x7f41001986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50966/0 (socket says 172.21.15.110:50966) 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.621+0000 7f4104b45700 1 -- 172.21.15.110:0/2121673477 learned_addr learned my addr 172.21.15.110:0/2121673477 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f40fffff700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4100101130 0x7f410019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f40ff7fe700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4100101af0 0x7f410019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f4104b45700 1 -- 172.21.15.110:0/2121673477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4100101af0 msgr2=0x7f410019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f4104b45700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4100101af0 0x7f410019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f4104b45700 1 -- 172.21.15.110:0/2121673477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4100101130 msgr2=0x7f410019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f4104b45700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4100101130 0x7f410019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f4104b45700 1 -- 172.21.15.110:0/2121673477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4100198f90 con 0x7f41001071a0 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f4104b45700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41001071a0 0x7f41001986b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009fd0 tx=0x7f40f8004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f40fd7fa700 1 -- 172.21.15.110:0/2121673477 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8021070 con 0x7f41001071a0 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f40fd7fa700 1 -- 172.21.15.110:0/2121673477 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40f8007c00 con 0x7f41001071a0 2024-04-25T00:01:16.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f40fd7fa700 1 -- 172.21.15.110:0/2121673477 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80052a0 con 0x7f41001071a0 2024-04-25T00:01:16.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.622+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4100199220 con 0x7f41001071a0 2024-04-25T00:01:16.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.623+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41001a5560 con 0x7f41001071a0 2024-04-25T00:01:16.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.623+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4100102850 con 0x7f41001071a0 2024-04-25T00:01:16.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.625+0000 7f40fd7fa700 1 -- 172.21.15.110:0/2121673477 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f40f8024070 con 0x7f41001071a0 2024-04-25T00:01:16.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.626+0000 7f40fd7fa700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2d30 0x7f40e80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:16.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.626+0000 7f40fffff700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2d30 0x7f40e80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:16.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.626+0000 7f40fd7fa700 1 -- 172.21.15.110:0/2121673477 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f40f800f020 con 0x7f41001071a0 2024-04-25T00:01:16.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.627+0000 7f40fffff700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2d30 0x7f40e80a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40f000abd0 tx=0x7f40f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:16.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.630+0000 7f40fd7fa700 1 -- 172.21.15.110:0/2121673477 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f40f802b020 con 0x7f41001071a0 2024-04-25T00:01:16.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.833+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f41001a5730 con 0x7f41001071a0 2024-04-25T00:01:16.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.833+0000 7f40fd7fa700 1 -- 172.21.15.110:0/2121673477 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40f80bbe20 con 0x7f41001071a0 2024-04-25T00:01:16.833 INFO:teuthology.orchestra.run.smithi110.stdout:77309411402 2024-04-25T00:01:16.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2d30 msgr2=0x7f40e80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2d30 0x7f40e80a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40f000abd0 tx=0x7f40f000a380 comp rx=0 tx=0).stop 2024-04-25T00:01:16.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41001071a0 msgr2=0x7f41001986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:16.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41001071a0 0x7f41001986b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009fd0 tx=0x7f40f8004dc0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 shutdown_connections 2024-04-25T00:01:16.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f40e80a2d30 0x7f40e80a51f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41001071a0 0x7f41001986b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4100101130 0x7f410019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 --2- 172.21.15.110:0/2121673477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4100101af0 0x7f410019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:16.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 >> 172.21.15.110:0/2121673477 conn(0x7f41000780e0 msgr2=0x7f41000fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:16.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 shutdown_connections 2024-04-25T00:01:16.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:16.835+0000 7f41065a8700 1 -- 172.21.15.110:0/2121673477 wait complete. 2024-04-25T00:01:16.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:16 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4159988161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:01:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:16 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4159988161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:01:17.430 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332139 got 219043332144 for osd.7 2024-04-25T00:01:17.430 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:17.738 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:17 smithi110 ceph-mon[29187]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:17.739 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:17 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2121673477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:01:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:17 smithi173 ceph-mon[32402]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:17.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:17 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2121673477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:01:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:17 smithi188 ceph-mon[33285]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:18.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:17 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2121673477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:01:18.087 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808868 for osd.9 2024-04-25T00:01:18.087 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:18.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 -- 172.21.15.110:0/3612176125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 msgr2=0x7f957c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 --2- 172.21.15.110:0/3612176125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 0x7f957c101f70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f956c0099c0 tx=0x7f956c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 -- 172.21.15.110:0/3612176125 shutdown_connections 2024-04-25T00:01:18.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 --2- 172.21.15.110:0/3612176125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c1064b0 0x7f957c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 --2- 172.21.15.110:0/3612176125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 0x7f957c101f70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 --2- 172.21.15.110:0/3612176125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957c102540 0x7f957c105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 -- 172.21.15.110:0/3612176125 >> 172.21.15.110:0/3612176125 conn(0x7f957c0777a0 msgr2=0x7f957c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 -- 172.21.15.110:0/3612176125 shutdown_connections 2024-04-25T00:01:18.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.379+0000 7f9581d20700 1 -- 172.21.15.110:0/3612176125 wait complete. 2024-04-25T00:01:18.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f9581d20700 1 Processor -- start 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f9581d20700 1 -- start start 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f9581d20700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 0x7f957c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f9581d20700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c102540 0x7f957c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f9581d20700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957c1064b0 0x7f957c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f9581d20700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f957c111400 con 0x7f957c101b90 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f9581d20700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f957c111280 con 0x7f957c102540 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f9581d20700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f957c111580 con 0x7f957c1064b0 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 0x7f957c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 0x7f957c19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50982/0 (socket says 172.21.15.110:50982) 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 -- 172.21.15.110:0/3674320994 learned_addr learned my addr 172.21.15.110:0/3674320994 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957bfff700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957c1064b0 0x7f957c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 -- 172.21.15.110:0/3674320994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957c1064b0 msgr2=0x7f957c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957c1064b0 0x7f957c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 -- 172.21.15.110:0/3674320994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c102540 msgr2=0x7f957c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c102540 0x7f957c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 -- 172.21.15.110:0/3674320994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c199030 con 0x7f957c101b90 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.380+0000 7f957b7fe700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 0x7f957c19e6c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f956c006010 tx=0x7f956c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.381+0000 7f9578ff9700 1 -- 172.21.15.110:0/3674320994 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c027070 con 0x7f957c101b90 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.381+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c1992c0 con 0x7f957c101b90 2024-04-25T00:01:18.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.381+0000 7f9578ff9700 1 -- 172.21.15.110:0/3674320994 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956c003fd0 con 0x7f957c101b90 2024-04-25T00:01:18.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.381+0000 7f9578ff9700 1 -- 172.21.15.110:0/3674320994 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c022d70 con 0x7f957c101b90 2024-04-25T00:01:18.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.381+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c1a5590 con 0x7f957c101b90 2024-04-25T00:01:18.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.382+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c102940 con 0x7f957c101b90 2024-04-25T00:01:18.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.383+0000 7f9578ff9700 1 -- 172.21.15.110:0/3674320994 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f956c02a070 con 0x7f957c101b90 2024-04-25T00:01:18.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.383+0000 7f9578ff9700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95640a2cb0 0x7f95640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:18.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.383+0000 7f957affd700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95640a2cb0 0x7f95640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:18.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.383+0000 7f9578ff9700 1 -- 172.21.15.110:0/3674320994 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f956c0cff40 con 0x7f957c101b90 2024-04-25T00:01:18.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.384+0000 7f957affd700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95640a2cb0 0x7f95640a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f957c103ea0 tx=0x7f9570006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:18.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.388+0000 7f9578ff9700 1 -- 172.21.15.110:0/3674320994 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f956c05e740 con 0x7f957c101b90 2024-04-25T00:01:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.605+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f957c068f20 con 0x7f957c101b90 2024-04-25T00:01:18.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.605+0000 7f9578ff9700 1 -- 172.21.15.110:0/3674320994 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f956c013070 con 0x7f957c101b90 2024-04-25T00:01:18.607 INFO:teuthology.orchestra.run.smithi110.stdout:253403070507 2024-04-25T00:01:18.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95640a2cb0 msgr2=0x7f95640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95640a2cb0 0x7f95640a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f957c103ea0 tx=0x7f9570006d20 comp rx=0 tx=0).stop 2024-04-25T00:01:18.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 msgr2=0x7f957c19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:18.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 0x7f957c19e6c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f956c006010 tx=0x7f956c009f80 comp rx=0 tx=0).stop 2024-04-25T00:01:18.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 shutdown_connections 2024-04-25T00:01:18.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f95640a2cb0 0x7f95640a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957c1064b0 0x7f957c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f957c101b90 0x7f957c19e6c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 --2- 172.21.15.110:0/3674320994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c102540 0x7f957c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:18.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 >> 172.21.15.110:0/3674320994 conn(0x7f957c0777a0 msgr2=0x7f957c100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:18.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 shutdown_connections 2024-04-25T00:01:18.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:18.608+0000 7f9581d20700 1 -- 172.21.15.110:0/3674320994 wait complete. 2024-04-25T00:01:18.892 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383701 got 343597383706 for osd.11 2024-04-25T00:01:18.893 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:19.559 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411402 for osd.2 2024-04-25T00:01:19.560 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:19.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:19 smithi110 ceph-mon[29187]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:19.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:19 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3674320994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:01:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:19 smithi173 ceph-mon[32402]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:19.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:19 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3674320994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:01:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:19 smithi188 ceph-mon[33285]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:20.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:19 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3674320994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:01:20.218 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070499 got 253403070507 for osd.8 2024-04-25T00:01:20.218 DEBUG:teuthology.parallel:result is None 2024-04-25T00:01:20.219 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T00:01:20.219 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:01:20.795 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:21.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:21 smithi173 ceph-mon[32402]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:21.956 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:21 smithi110 ceph-mon[29187]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:21 smithi188 ceph-mon[33285]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.550+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/2438756560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 msgr2=0x7f5b48106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.550+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/2438756560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 0x7f5b48106e90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009990 tx=0x7f5b38009ca0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/2438756560 shutdown_connections 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/2438756560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b48107570 0x7f5b4810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/2438756560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b481024e0 0x7f5b481028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/2438756560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 0x7f5b48106e90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/2438756560 >> 172.21.15.110:0/2438756560 conn(0x7f5b48079390 msgr2=0x7f5b480797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/2438756560 shutdown_connections 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/2438756560 wait complete. 2024-04-25T00:01:22.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 Processor -- start 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.551+0000 7f5b4cd5d700 1 -- start start 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b4cd5d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b481024e0 0x7f5b4819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b4cd5d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 0x7f5b4819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b4cd5d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b48107570 0x7f5b48198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b4cd5d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481123e0 con 0x7f5b48102e90 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b4cd5d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48112260 con 0x7f5b48107570 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b4cd5d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48112560 con 0x7f5b481024e0 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 0x7f5b4819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 0x7f5b4819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51000/0 (socket says 172.21.15.110:51000) 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 -- 172.21.15.110:0/3689371892 learned_addr learned my addr 172.21.15.110:0/3689371892 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b4659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b481024e0 0x7f5b4819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b46d9d700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b48107570 0x7f5b48198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 -- 172.21.15.110:0/3689371892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b481024e0 msgr2=0x7f5b4819e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b481024e0 0x7f5b4819e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 -- 172.21.15.110:0/3689371892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b48107570 msgr2=0x7f5b48198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b48107570 0x7f5b48198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 -- 172.21.15.110:0/3689371892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b48198f60 con 0x7f5b48102e90 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b45d9b700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 0x7f5b4819eb40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c00 tx=0x7f5b38003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b3f7fe700 1 -- 172.21.15.110:0/3689371892 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38027070 con 0x7f5b48102e90 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.552+0000 7f5b3f7fe700 1 -- 172.21.15.110:0/3689371892 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b3800bdc0 con 0x7f5b48102e90 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.553+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b481991f0 con 0x7f5b48102e90 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.553+0000 7f5b3f7fe700 1 -- 172.21.15.110:0/3689371892 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38022ca0 con 0x7f5b48102e90 2024-04-25T00:01:22.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.553+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b481a5460 con 0x7f5b48102e90 2024-04-25T00:01:22.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.553+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b48103970 con 0x7f5b48102e90 2024-04-25T00:01:22.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.555+0000 7f5b3f7fe700 1 -- 172.21.15.110:0/3689371892 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5b3802a070 con 0x7f5b48102e90 2024-04-25T00:01:22.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.555+0000 7f5b3f7fe700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b340a2cb0 0x7f5b340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:22.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.555+0000 7f5b4659c700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b340a2cb0 0x7f5b340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:22.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.555+0000 7f5b3f7fe700 1 -- 172.21.15.110:0/3689371892 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5b380d02a0 con 0x7f5b48102e90 2024-04-25T00:01:22.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.556+0000 7f5b4659c700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b340a2cb0 0x7f5b340a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b30004630 tx=0x7f5b3000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:22.558 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.559+0000 7f5b3f7fe700 1 -- 172.21.15.110:0/3689371892 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5b38013070 con 0x7f5b48102e90 2024-04-25T00:01:22.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.775+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b48199cb0 con 0x7f5b340a2cb0 2024-04-25T00:01:22.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.783+0000 7f5b3f7fe700 1 -- 172.21.15.110:0/3689371892 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258930 (secure 0 0 0) 0x7f5b48199cb0 con 0x7f5b340a2cb0 2024-04-25T00:01:22.787 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:22.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b340a2cb0 msgr2=0x7f5b340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b340a2cb0 0x7f5b340a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b30004630 tx=0x7f5b3000b3c0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 msgr2=0x7f5b4819eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:22.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 0x7f5b4819eb40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c00 tx=0x7f5b38003a80 comp rx=0 tx=0).stop 2024-04-25T00:01:22.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 shutdown_connections 2024-04-25T00:01:22.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5b340a2cb0 0x7f5b340a5170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b48107570 0x7f5b48198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b481024e0 0x7f5b4819e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 --2- 172.21.15.110:0/3689371892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b48102e90 0x7f5b4819eb40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:22.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.786+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 >> 172.21.15.110:0/3689371892 conn(0x7f5b48079390 msgr2=0x7f5b481086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:22.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.787+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 shutdown_connections 2024-04-25T00:01:22.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:22.787+0000 7f5b4cd5d700 1 -- 172.21.15.110:0/3689371892 wait complete. 2024-04-25T00:01:22.790 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-04-25T00:01:23.528 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-04-25T00:01:22.527919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653760,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":584,"num_read_kb":399,"num_write":340,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352468,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589356,"statfs":{"total":1151940427776,"available":1151579500544,"internally_reserved":0,"allocated":11833344,"data_stored":7427520,"data_compressed":113994,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002909"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654019+0000","last_change":"2024-04-25T00:00:03.143150+0000","last_active":"2024-04-25T00:00:55.654019+0000","last_peered":"2024-04-25T00:00:55.654019+0000","last_clean":"2024-04-25T00:00:55.654019+0000","last_became_active":"2024-04-25T00:00:03.142344+0000","last_became_peered":"2024-04-25T00:00:03.142344+0000","last_unstale":"2024-04-25T00:00:55.654019+0000","last_undegraded":"2024-04-25T00:00:55.654019+0000","last_fullsized":"2024-04-25T00:00:55.654019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:18.293748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651560+0000","last_change":"2024-04-25T00:00:03.145454+0000","last_active":"2024-04-25T00:00:55.651560+0000","last_peered":"2024-04-25T00:00:55.651560+0000","last_clean":"2024-04-25T00:00:55.651560+0000","last_became_active":"2024-04-25T00:00:03.144905+0000","last_became_peered":"2024-04-25T00:00:03.144905+0000","last_unstale":"2024-04-25T00:00:55.651560+0000","last_undegraded":"2024-04-25T00:00:55.651560+0000","last_fullsized":"2024-04-25T00:00:55.651560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:39.583313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651373+0000","last_change":"2024-04-25T00:00:03.141770+0000","last_active":"2024-04-25T00:00:55.651373+0000","last_peered":"2024-04-25T00:00:55.651373+0000","last_clean":"2024-04-25T00:00:55.651373+0000","last_became_active":"2024-04-25T00:00:03.141468+0000","last_became_peered":"2024-04-25T00:00:03.141468+0000","last_unstale":"2024-04-25T00:00:55.651373+0000","last_undegraded":"2024-04-25T00:00:55.651373+0000","last_fullsized":"2024-04-25T00:00:55.651373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:33.112112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.660753+0000","last_change":"2024-04-25T00:00:55.660277+0000","last_active":"2024-04-25T00:00:55.660753+0000","last_peered":"2024-04-25T00:00:55.660753+0000","last_clean":"2024-04-25T00:00:55.660753+0000","last_became_active":"2024-04-25T00:00:55.659851+0000","last_became_peered":"2024-04-25T00:00:55.659851+0000","last_unstale":"2024-04-25T00:00:55.660753+0000","last_undegraded":"2024-04-25T00:00:55.660753+0000","last_fullsized":"2024-04-25T00:00:55.660753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:04.471838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652056+0000","last_change":"2024-04-25T00:00:03.146405+0000","last_active":"2024-04-25T00:00:55.652056+0000","last_peered":"2024-04-25T00:00:55.652056+0000","last_clean":"2024-04-25T00:00:55.652056+0000","last_became_active":"2024-04-25T00:00:03.145520+0000","last_became_peered":"2024-04-25T00:00:03.145520+0000","last_unstale":"2024-04-25T00:00:55.652056+0000","last_undegraded":"2024-04-25T00:00:55.652056+0000","last_fullsized":"2024-04-25T00:00:55.652056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:02.256905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017163+0000","last_change":"2024-04-25T00:00:03.142973+0000","last_active":"2024-04-25T00:00:56.017163+0000","last_peered":"2024-04-25T00:00:56.017163+0000","last_clean":"2024-04-25T00:00:56.017163+0000","last_became_active":"2024-04-25T00:00:03.142091+0000","last_became_peered":"2024-04-25T00:00:03.142091+0000","last_unstale":"2024-04-25T00:00:56.017163+0000","last_undegraded":"2024-04-25T00:00:56.017163+0000","last_fullsized":"2024-04-25T00:00:56.017163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:01.948299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653849+0000","last_change":"2024-04-25T00:00:03.148383+0000","last_active":"2024-04-25T00:00:55.653849+0000","last_peered":"2024-04-25T00:00:55.653849+0000","last_clean":"2024-04-25T00:00:55.653849+0000","last_became_active":"2024-04-25T00:00:03.148105+0000","last_became_peered":"2024-04-25T00:00:03.148105+0000","last_unstale":"2024-04-25T00:00:55.653849+0000","last_undegraded":"2024-04-25T00:00:55.653849+0000","last_fullsized":"2024-04-25T00:00:55.653849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:31.902600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656094+0000","last_change":"2024-04-25T00:00:03.151575+0000","last_active":"2024-04-25T00:00:55.656094+0000","last_peered":"2024-04-25T00:00:55.656094+0000","last_clean":"2024-04-25T00:00:55.656094+0000","last_became_active":"2024-04-25T00:00:03.151269+0000","last_became_peered":"2024-04-25T00:00:03.151269+0000","last_unstale":"2024-04-25T00:00:55.656094+0000","last_undegraded":"2024-04-25T00:00:55.656094+0000","last_fullsized":"2024-04-25T00:00:55.656094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:16.561180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657083+0000","last_change":"2024-04-25T00:00:03.140175+0000","last_active":"2024-04-25T00:00:55.657083+0000","last_peered":"2024-04-25T00:00:55.657083+0000","last_clean":"2024-04-25T00:00:55.657083+0000","last_became_active":"2024-04-25T00:00:03.139838+0000","last_became_peered":"2024-04-25T00:00:03.139838+0000","last_unstale":"2024-04-25T00:00:55.657083+0000","last_undegraded":"2024-04-25T00:00:55.657083+0000","last_fullsized":"2024-04-25T00:00:55.657083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:16.768025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017489+0000","last_change":"2024-04-25T00:00:03.143149+0000","last_active":"2024-04-25T00:00:56.017489+0000","last_peered":"2024-04-25T00:00:56.017489+0000","last_clean":"2024-04-25T00:00:56.017489+0000","last_became_active":"2024-04-25T00:00:03.142799+0000","last_became_peered":"2024-04-25T00:00:03.142799+0000","last_unstale":"2024-04-25T00:00:56.017489+0000","last_undegraded":"2024-04-25T00:00:56.017489+0000","last_fullsized":"2024-04-25T00:00:56.017489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:38.941329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654510+0000","last_change":"2024-04-25T00:00:03.141519+0000","last_active":"2024-04-25T00:00:55.654510+0000","last_peered":"2024-04-25T00:00:55.654510+0000","last_clean":"2024-04-25T00:00:55.654510+0000","last_became_active":"2024-04-25T00:00:03.140452+0000","last_became_peered":"2024-04-25T00:00:03.140452+0000","last_unstale":"2024-04-25T00:00:55.654510+0000","last_undegraded":"2024-04-25T00:00:55.654510+0000","last_fullsized":"2024-04-25T00:00:55.654510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:17.552403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653419+0000","last_change":"2024-04-25T00:00:03.149727+0000","last_active":"2024-04-25T00:00:55.653419+0000","last_peered":"2024-04-25T00:00:55.653419+0000","last_clean":"2024-04-25T00:00:55.653419+0000","last_became_active":"2024-04-25T00:00:03.149205+0000","last_became_peered":"2024-04-25T00:00:03.149205+0000","last_unstale":"2024-04-25T00:00:55.653419+0000","last_undegraded":"2024-04-25T00:00:55.653419+0000","last_fullsized":"2024-04-25T00:00:55.653419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:55.782750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654313+0000","last_change":"2024-04-25T00:00:03.139219+0000","last_active":"2024-04-25T00:00:55.654313+0000","last_peered":"2024-04-25T00:00:55.654313+0000","last_clean":"2024-04-25T00:00:55.654313+0000","last_became_active":"2024-04-25T00:00:03.138883+0000","last_became_peered":"2024-04-25T00:00:03.138883+0000","last_unstale":"2024-04-25T00:00:55.654313+0000","last_undegraded":"2024-04-25T00:00:55.654313+0000","last_fullsized":"2024-04-25T00:00:55.654313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.744932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651272+0000","last_change":"2024-04-25T00:00:03.144473+0000","last_active":"2024-04-25T00:00:55.651272+0000","last_peered":"2024-04-25T00:00:55.651272+0000","last_clean":"2024-04-25T00:00:55.651272+0000","last_became_active":"2024-04-25T00:00:03.144166+0000","last_became_peered":"2024-04-25T00:00:03.144166+0000","last_unstale":"2024-04-25T00:00:55.651272+0000","last_undegraded":"2024-04-25T00:00:55.651272+0000","last_fullsized":"2024-04-25T00:00:55.651272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:25.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651170+0000","last_change":"2024-04-25T00:00:03.143292+0000","last_active":"2024-04-25T00:00:55.651170+0000","last_peered":"2024-04-25T00:00:55.651170+0000","last_clean":"2024-04-25T00:00:55.651170+0000","last_became_active":"2024-04-25T00:00:03.142993+0000","last_became_peered":"2024-04-25T00:00:03.142993+0000","last_unstale":"2024-04-25T00:00:55.651170+0000","last_undegraded":"2024-04-25T00:00:55.651170+0000","last_fullsized":"2024-04-25T00:00:55.651170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.841015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656227+0000","last_change":"2024-04-25T00:00:03.148117+0000","last_active":"2024-04-25T00:00:55.656227+0000","last_peered":"2024-04-25T00:00:55.656227+0000","last_clean":"2024-04-25T00:00:55.656227+0000","last_became_active":"2024-04-25T00:00:03.147670+0000","last_became_peered":"2024-04-25T00:00:03.147670+0000","last_unstale":"2024-04-25T00:00:55.656227+0000","last_undegraded":"2024-04-25T00:00:55.656227+0000","last_fullsized":"2024-04-25T00:00:55.656227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:01.716253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654118+0000","last_change":"2024-04-25T00:00:03.143452+0000","last_active":"2024-04-25T00:00:55.654118+0000","last_peered":"2024-04-25T00:00:55.654118+0000","last_clean":"2024-04-25T00:00:55.654118+0000","last_became_active":"2024-04-25T00:00:03.142379+0000","last_became_peered":"2024-04-25T00:00:03.142379+0000","last_unstale":"2024-04-25T00:00:55.654118+0000","last_undegraded":"2024-04-25T00:00:55.654118+0000","last_fullsized":"2024-04-25T00:00:55.654118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:46.828363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654235+0000","last_change":"2024-04-25T00:00:03.138570+0000","last_active":"2024-04-25T00:00:55.654235+0000","last_peered":"2024-04-25T00:00:55.654235+0000","last_clean":"2024-04-25T00:00:55.654235+0000","last_became_active":"2024-04-25T00:00:03.138198+0000","last_became_peered":"2024-04-25T00:00:03.138198+0000","last_unstale":"2024-04-25T00:00:55.654235+0000","last_undegraded":"2024-04-25T00:00:55.654235+0000","last_fullsized":"2024-04-25T00:00:55.654235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:47.794135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656684+0000","last_change":"2024-04-25T00:00:03.143152+0000","last_active":"2024-04-25T00:00:55.656684+0000","last_peered":"2024-04-25T00:00:55.656684+0000","last_clean":"2024-04-25T00:00:55.656684+0000","last_became_active":"2024-04-25T00:00:03.142812+0000","last_became_peered":"2024-04-25T00:00:03.142812+0000","last_unstale":"2024-04-25T00:00:55.656684+0000","last_undegraded":"2024-04-25T00:00:55.656684+0000","last_fullsized":"2024-04-25T00:00:55.656684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:58.808783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654008+0000","last_change":"2024-04-25T00:00:02.525740+0000","last_active":"2024-04-25T00:00:55.654008+0000","last_peered":"2024-04-25T00:00:55.654008+0000","last_clean":"2024-04-25T00:00:55.654008+0000","last_became_active":"2024-04-25T00:00:02.525429+0000","last_became_peered":"2024-04-25T00:00:02.525429+0000","last_unstale":"2024-04-25T00:00:55.654008+0000","last_undegraded":"2024-04-25T00:00:55.654008+0000","last_fullsized":"2024-04-25T00:00:55.654008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:43.585678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.660547+0000","last_change":"2024-04-25T00:00:55.660072+0000","last_active":"2024-04-25T00:00:55.660547+0000","last_peered":"2024-04-25T00:00:55.660547+0000","last_clean":"2024-04-25T00:00:55.660547+0000","last_became_active":"2024-04-25T00:00:55.659804+0000","last_became_peered":"2024-04-25T00:00:55.659804+0000","last_unstale":"2024-04-25T00:00:55.660547+0000","last_undegraded":"2024-04-25T00:00:55.660547+0000","last_fullsized":"2024-04-25T00:00:55.660547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:52.780468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656749+0000","last_change":"2024-04-25T00:00:02.534181+0000","last_active":"2024-04-25T00:00:55.656749+0000","last_peered":"2024-04-25T00:00:55.656749+0000","last_clean":"2024-04-25T00:00:55.656749+0000","last_became_active":"2024-04-25T00:00:02.533848+0000","last_became_peered":"2024-04-25T00:00:02.533848+0000","last_unstale":"2024-04-25T00:00:55.656749+0000","last_undegraded":"2024-04-25T00:00:55.656749+0000","last_fullsized":"2024-04-25T00:00:55.656749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:25.276477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651607+0000","last_change":"2024-04-25T00:00:03.142038+0000","last_active":"2024-04-25T00:00:55.651607+0000","last_peered":"2024-04-25T00:00:55.651607+0000","last_clean":"2024-04-25T00:00:55.651607+0000","last_became_active":"2024-04-25T00:00:03.141705+0000","last_became_peered":"2024-04-25T00:00:03.141705+0000","last_unstale":"2024-04-25T00:00:55.651607+0000","last_undegraded":"2024-04-25T00:00:55.651607+0000","last_fullsized":"2024-04-25T00:00:55.651607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.382031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652506+0000","last_change":"2024-04-25T00:00:02.526339+0000","last_active":"2024-04-25T00:00:55.652506+0000","last_peered":"2024-04-25T00:00:55.652506+0000","last_clean":"2024-04-25T00:00:55.652506+0000","last_became_active":"2024-04-25T00:00:02.525999+0000","last_became_peered":"2024-04-25T00:00:02.525999+0000","last_unstale":"2024-04-25T00:00:55.652506+0000","last_undegraded":"2024-04-25T00:00:55.652506+0000","last_fullsized":"2024-04-25T00:00:55.652506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:52.997821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656619+0000","last_change":"2024-04-25T00:00:03.143896+0000","last_active":"2024-04-25T00:00:55.656619+0000","last_peered":"2024-04-25T00:00:55.656619+0000","last_clean":"2024-04-25T00:00:55.656619+0000","last_became_active":"2024-04-25T00:00:03.141311+0000","last_became_peered":"2024-04-25T00:00:03.141311+0000","last_unstale":"2024-04-25T00:00:55.656619+0000","last_undegraded":"2024-04-25T00:00:55.656619+0000","last_fullsized":"2024-04-25T00:00:55.656619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:57.819319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.666658+0000","last_change":"2024-04-25T00:00:55.666220+0000","last_active":"2024-04-25T00:00:55.666658+0000","last_peered":"2024-04-25T00:00:55.666658+0000","last_clean":"2024-04-25T00:00:55.666658+0000","last_became_active":"2024-04-25T00:00:55.659538+0000","last_became_peered":"2024-04-25T00:00:55.659538+0000","last_unstale":"2024-04-25T00:00:55.666658+0000","last_undegraded":"2024-04-25T00:00:55.666658+0000","last_fullsized":"2024-04-25T00:00:55.666658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:05.888688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657973+0000","last_change":"2024-04-25T00:00:03.140867+0000","last_active":"2024-04-25T00:00:55.657973+0000","last_peered":"2024-04-25T00:00:55.657973+0000","last_clean":"2024-04-25T00:00:55.657973+0000","last_became_active":"2024-04-25T00:00:03.140545+0000","last_became_peered":"2024-04-25T00:00:03.140545+0000","last_unstale":"2024-04-25T00:00:55.657973+0000","last_undegraded":"2024-04-25T00:00:55.657973+0000","last_fullsized":"2024-04-25T00:00:55.657973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:46.928359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:20.668943+0000","last_change":"2024-04-25T00:00:02.528897+0000","last_active":"2024-04-25T00:01:20.668943+0000","last_peered":"2024-04-25T00:01:20.668943+0000","last_clean":"2024-04-25T00:01:20.668943+0000","last_became_active":"2024-04-25T00:00:02.528584+0000","last_became_peered":"2024-04-25T00:00:02.528584+0000","last_unstale":"2024-04-25T00:01:20.668943+0000","last_undegraded":"2024-04-25T00:01:20.668943+0000","last_fullsized":"2024-04-25T00:01:20.668943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:23.437956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651937+0000","last_change":"2024-04-25T00:00:03.147692+0000","last_active":"2024-04-25T00:00:55.651937+0000","last_peered":"2024-04-25T00:00:55.651937+0000","last_clean":"2024-04-25T00:00:55.651937+0000","last_became_active":"2024-04-25T00:00:03.147183+0000","last_became_peered":"2024-04-25T00:00:03.147183+0000","last_unstale":"2024-04-25T00:00:55.651937+0000","last_undegraded":"2024-04-25T00:00:55.651937+0000","last_fullsized":"2024-04-25T00:00:55.651937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:12.640665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017105+0000","last_change":"2024-04-25T00:00:02.522387+0000","last_active":"2024-04-25T00:00:56.017105+0000","last_peered":"2024-04-25T00:00:56.017105+0000","last_clean":"2024-04-25T00:00:56.017105+0000","last_became_active":"2024-04-25T00:00:02.522093+0000","last_became_peered":"2024-04-25T00:00:02.522093+0000","last_unstale":"2024-04-25T00:00:56.017105+0000","last_undegraded":"2024-04-25T00:00:56.017105+0000","last_fullsized":"2024-04-25T00:00:56.017105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:33.393915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657355+0000","last_change":"2024-04-25T00:00:03.146277+0000","last_active":"2024-04-25T00:00:55.657355+0000","last_peered":"2024-04-25T00:00:55.657355+0000","last_clean":"2024-04-25T00:00:55.657355+0000","last_became_active":"2024-04-25T00:00:03.145846+0000","last_became_peered":"2024-04-25T00:00:03.145846+0000","last_unstale":"2024-04-25T00:00:55.657355+0000","last_undegraded":"2024-04-25T00:00:55.657355+0000","last_fullsized":"2024-04-25T00:00:55.657355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.692246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654117+0000","last_change":"2024-04-25T00:00:02.528405+0000","last_active":"2024-04-25T00:00:55.654117+0000","last_peered":"2024-04-25T00:00:55.654117+0000","last_clean":"2024-04-25T00:00:55.654117+0000","last_became_active":"2024-04-25T00:00:02.528115+0000","last_became_peered":"2024-04-25T00:00:02.528115+0000","last_unstale":"2024-04-25T00:00:55.654117+0000","last_undegraded":"2024-04-25T00:00:55.654117+0000","last_fullsized":"2024-04-25T00:00:55.654117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:51.382361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.655982+0000","last_change":"2024-04-25T00:00:03.144266+0000","last_active":"2024-04-25T00:00:55.655982+0000","last_peered":"2024-04-25T00:00:55.655982+0000","last_clean":"2024-04-25T00:00:55.655982+0000","last_became_active":"2024-04-25T00:00:03.141704+0000","last_became_peered":"2024-04-25T00:00:03.141704+0000","last_unstale":"2024-04-25T00:00:55.655982+0000","last_undegraded":"2024-04-25T00:00:55.655982+0000","last_fullsized":"2024-04-25T00:00:55.655982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:14.161783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657192+0000","last_change":"2024-04-25T00:00:02.528134+0000","last_active":"2024-04-25T00:00:55.657192+0000","last_peered":"2024-04-25T00:00:55.657192+0000","last_clean":"2024-04-25T00:00:55.657192+0000","last_became_active":"2024-04-25T00:00:02.527830+0000","last_became_peered":"2024-04-25T00:00:02.527830+0000","last_unstale":"2024-04-25T00:00:55.657192+0000","last_undegraded":"2024-04-25T00:00:55.657192+0000","last_fullsized":"2024-04-25T00:00:55.657192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:46.846244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653991+0000","last_change":"2024-04-25T00:00:03.143505+0000","last_active":"2024-04-25T00:00:55.653991+0000","last_peered":"2024-04-25T00:00:55.653991+0000","last_clean":"2024-04-25T00:00:55.653991+0000","last_became_active":"2024-04-25T00:00:03.143131+0000","last_became_peered":"2024-04-25T00:00:03.143131+0000","last_unstale":"2024-04-25T00:00:55.653991+0000","last_undegraded":"2024-04-25T00:00:55.653991+0000","last_fullsized":"2024-04-25T00:00:55.653991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:23.743162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:15.660853+0000","last_change":"2024-04-25T00:00:02.525978+0000","last_active":"2024-04-25T00:01:15.660853+0000","last_peered":"2024-04-25T00:01:15.660853+0000","last_clean":"2024-04-25T00:01:15.660853+0000","last_became_active":"2024-04-25T00:00:02.525641+0000","last_became_peered":"2024-04-25T00:00:02.525641+0000","last_unstale":"2024-04-25T00:01:15.660853+0000","last_undegraded":"2024-04-25T00:01:15.660853+0000","last_fullsized":"2024-04-25T00:01:15.660853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:58.508573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656866+0000","last_change":"2024-04-25T00:00:03.143734+0000","last_active":"2024-04-25T00:00:55.656866+0000","last_peered":"2024-04-25T00:00:55.656866+0000","last_clean":"2024-04-25T00:00:55.656866+0000","last_became_active":"2024-04-25T00:00:03.143402+0000","last_became_peered":"2024-04-25T00:00:03.143402+0000","last_unstale":"2024-04-25T00:00:55.656866+0000","last_undegraded":"2024-04-25T00:00:55.656866+0000","last_fullsized":"2024-04-25T00:00:55.656866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:36.608803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654913+0000","last_change":"2024-04-25T00:00:02.530567+0000","last_active":"2024-04-25T00:00:55.654913+0000","last_peered":"2024-04-25T00:00:55.654913+0000","last_clean":"2024-04-25T00:00:55.654913+0000","last_became_active":"2024-04-25T00:00:02.530066+0000","last_became_peered":"2024-04-25T00:00:02.530066+0000","last_unstale":"2024-04-25T00:00:55.654913+0000","last_undegraded":"2024-04-25T00:00:55.654913+0000","last_fullsized":"2024-04-25T00:00:55.654913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:19.892052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651530+0000","last_change":"2024-04-25T00:00:03.146905+0000","last_active":"2024-04-25T00:00:55.651530+0000","last_peered":"2024-04-25T00:00:55.651530+0000","last_clean":"2024-04-25T00:00:55.651530+0000","last_became_active":"2024-04-25T00:00:03.146080+0000","last_became_peered":"2024-04-25T00:00:03.146080+0000","last_unstale":"2024-04-25T00:00:55.651530+0000","last_undegraded":"2024-04-25T00:00:55.651530+0000","last_fullsized":"2024-04-25T00:00:55.651530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:42.764082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.658095+0000","last_change":"2024-04-25T00:00:02.527667+0000","last_active":"2024-04-25T00:00:55.658095+0000","last_peered":"2024-04-25T00:00:55.658095+0000","last_clean":"2024-04-25T00:00:55.658095+0000","last_became_active":"2024-04-25T00:00:02.527333+0000","last_became_peered":"2024-04-25T00:00:02.527333+0000","last_unstale":"2024-04-25T00:00:55.658095+0000","last_undegraded":"2024-04-25T00:00:55.658095+0000","last_fullsized":"2024-04-25T00:00:55.658095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:15.982704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656832+0000","last_change":"2024-04-25T00:00:03.145856+0000","last_active":"2024-04-25T00:00:55.656832+0000","last_peered":"2024-04-25T00:00:55.656832+0000","last_clean":"2024-04-25T00:00:55.656832+0000","last_became_active":"2024-04-25T00:00:03.145406+0000","last_became_peered":"2024-04-25T00:00:03.145406+0000","last_unstale":"2024-04-25T00:00:55.656832+0000","last_undegraded":"2024-04-25T00:00:55.656832+0000","last_fullsized":"2024-04-25T00:00:55.656832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:28.318575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":195,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:16.662234+0000","last_change":"2024-04-25T00:00:02.522294+0000","last_active":"2024-04-25T00:01:16.662234+0000","last_peered":"2024-04-25T00:01:16.662234+0000","last_clean":"2024-04-25T00:01:16.662234+0000","last_became_active":"2024-04-25T00:00:02.521776+0000","last_became_peered":"2024-04-25T00:00:02.521776+0000","last_unstale":"2024-04-25T00:01:16.662234+0000","last_undegraded":"2024-04-25T00:01:16.662234+0000","last_fullsized":"2024-04-25T00:01:16.662234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:07.677118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58766,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":148,"num_read_kb":130,"num_write":29,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652252+0000","last_change":"2024-04-25T00:00:03.143877+0000","last_active":"2024-04-25T00:00:55.652252+0000","last_peered":"2024-04-25T00:00:55.652252+0000","last_clean":"2024-04-25T00:00:55.652252+0000","last_became_active":"2024-04-25T00:00:03.143558+0000","last_became_peered":"2024-04-25T00:00:03.143558+0000","last_unstale":"2024-04-25T00:00:55.652252+0000","last_undegraded":"2024-04-25T00:00:55.652252+0000","last_fullsized":"2024-04-25T00:00:55.652252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:37.306175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656397+0000","last_change":"2024-04-25T00:00:02.520760+0000","last_active":"2024-04-25T00:00:55.656397+0000","last_peered":"2024-04-25T00:00:55.656397+0000","last_clean":"2024-04-25T00:00:55.656397+0000","last_became_active":"2024-04-25T00:00:02.520417+0000","last_became_peered":"2024-04-25T00:00:02.520417+0000","last_unstale":"2024-04-25T00:00:55.656397+0000","last_undegraded":"2024-04-25T00:00:55.656397+0000","last_fullsized":"2024-04-25T00:00:55.656397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:53.895653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654534+0000","last_change":"2024-04-25T00:00:03.144157+0000","last_active":"2024-04-25T00:00:55.654534+0000","last_peered":"2024-04-25T00:00:55.654534+0000","last_clean":"2024-04-25T00:00:55.654534+0000","last_became_active":"2024-04-25T00:00:03.143816+0000","last_became_peered":"2024-04-25T00:00:03.143816+0000","last_unstale":"2024-04-25T00:00:55.654534+0000","last_undegraded":"2024-04-25T00:00:55.654534+0000","last_fullsized":"2024-04-25T00:00:55.654534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:45.007842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657177+0000","last_change":"2024-04-25T00:00:02.534393+0000","last_active":"2024-04-25T00:00:55.657177+0000","last_peered":"2024-04-25T00:00:55.657177+0000","last_clean":"2024-04-25T00:00:55.657177+0000","last_became_active":"2024-04-25T00:00:02.534045+0000","last_became_peered":"2024-04-25T00:00:02.534045+0000","last_unstale":"2024-04-25T00:00:55.657177+0000","last_undegraded":"2024-04-25T00:00:55.657177+0000","last_fullsized":"2024-04-25T00:00:55.657177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:35.547065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657375+0000","last_change":"2024-04-25T00:00:55.657375+0000","last_active":"2024-04-25T00:00:55.657375+0000","last_peered":"2024-04-25T00:00:55.657375+0000","last_clean":"2024-04-25T00:00:55.657375+0000","last_became_active":"2024-04-25T00:00:55.656760+0000","last_became_peered":"2024-04-25T00:00:55.656760+0000","last_unstale":"2024-04-25T00:00:55.657375+0000","last_undegraded":"2024-04-25T00:00:55.657375+0000","last_fullsized":"2024-04-25T00:00:55.657375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:34.296182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":101,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:20.685993+0000","last_change":"2024-04-25T00:00:02.520357+0000","last_active":"2024-04-25T00:01:20.685993+0000","last_peered":"2024-04-25T00:01:20.685993+0000","last_clean":"2024-04-25T00:01:20.685993+0000","last_became_active":"2024-04-25T00:00:02.519926+0000","last_became_peered":"2024-04-25T00:00:02.519926+0000","last_unstale":"2024-04-25T00:01:20.685993+0000","last_undegraded":"2024-04-25T00:01:20.685993+0000","last_fullsized":"2024-04-25T00:01:20.685993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:52.438584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653891+0000","last_change":"2024-04-25T00:00:03.147560+0000","last_active":"2024-04-25T00:00:55.653891+0000","last_peered":"2024-04-25T00:00:55.653891+0000","last_clean":"2024-04-25T00:00:55.653891+0000","last_became_active":"2024-04-25T00:00:03.147234+0000","last_became_peered":"2024-04-25T00:00:03.147234+0000","last_unstale":"2024-04-25T00:00:55.653891+0000","last_undegraded":"2024-04-25T00:00:55.653891+0000","last_fullsized":"2024-04-25T00:00:55.653891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:33.737763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651858+0000","last_change":"2024-04-25T00:00:02.528614+0000","last_active":"2024-04-25T00:00:55.651858+0000","last_peered":"2024-04-25T00:00:55.651858+0000","last_clean":"2024-04-25T00:00:55.651858+0000","last_became_active":"2024-04-25T00:00:02.528203+0000","last_became_peered":"2024-04-25T00:00:02.528203+0000","last_unstale":"2024-04-25T00:00:55.651858+0000","last_undegraded":"2024-04-25T00:00:55.651858+0000","last_fullsized":"2024-04-25T00:00:55.651858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:33.116699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656908+0000","last_change":"2024-04-25T00:00:03.145009+0000","last_active":"2024-04-25T00:00:55.656908+0000","last_peered":"2024-04-25T00:00:55.656908+0000","last_clean":"2024-04-25T00:00:55.656908+0000","last_became_active":"2024-04-25T00:00:03.144575+0000","last_became_peered":"2024-04-25T00:00:03.144575+0000","last_unstale":"2024-04-25T00:00:55.656908+0000","last_undegraded":"2024-04-25T00:00:55.656908+0000","last_fullsized":"2024-04-25T00:00:55.656908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.219880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652436+0000","last_change":"2024-04-25T00:00:02.524951+0000","last_active":"2024-04-25T00:00:55.652436+0000","last_peered":"2024-04-25T00:00:55.652436+0000","last_clean":"2024-04-25T00:00:55.652436+0000","last_became_active":"2024-04-25T00:00:02.524624+0000","last_became_peered":"2024-04-25T00:00:02.524624+0000","last_unstale":"2024-04-25T00:00:55.652436+0000","last_undegraded":"2024-04-25T00:00:55.652436+0000","last_fullsized":"2024-04-25T00:00:55.652436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:23.381521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653872+0000","last_change":"2024-04-25T00:00:03.145446+0000","last_active":"2024-04-25T00:00:55.653872+0000","last_peered":"2024-04-25T00:00:55.653872+0000","last_clean":"2024-04-25T00:00:55.653872+0000","last_became_active":"2024-04-25T00:00:03.145125+0000","last_became_peered":"2024-04-25T00:00:03.145125+0000","last_unstale":"2024-04-25T00:00:55.653872+0000","last_undegraded":"2024-04-25T00:00:55.653872+0000","last_fullsized":"2024-04-25T00:00:55.653872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:40.187270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654370+0000","last_change":"2024-04-25T00:00:02.527499+0000","last_active":"2024-04-25T00:00:55.654370+0000","last_peered":"2024-04-25T00:00:55.654370+0000","last_clean":"2024-04-25T00:00:55.654370+0000","last_became_active":"2024-04-25T00:00:02.527151+0000","last_became_peered":"2024-04-25T00:00:02.527151+0000","last_unstale":"2024-04-25T00:00:55.654370+0000","last_undegraded":"2024-04-25T00:00:55.654370+0000","last_fullsized":"2024-04-25T00:00:55.654370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:09.013930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017381+0000","last_change":"2024-04-25T00:00:03.148231+0000","last_active":"2024-04-25T00:00:56.017381+0000","last_peered":"2024-04-25T00:00:56.017381+0000","last_clean":"2024-04-25T00:00:56.017381+0000","last_became_active":"2024-04-25T00:00:03.147891+0000","last_became_peered":"2024-04-25T00:00:03.147891+0000","last_unstale":"2024-04-25T00:00:56.017381+0000","last_undegraded":"2024-04-25T00:00:56.017381+0000","last_fullsized":"2024-04-25T00:00:56.017381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:42.784699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654156+0000","last_change":"2024-04-25T00:00:02.528234+0000","last_active":"2024-04-25T00:00:55.654156+0000","last_peered":"2024-04-25T00:00:55.654156+0000","last_clean":"2024-04-25T00:00:55.654156+0000","last_became_active":"2024-04-25T00:00:02.527875+0000","last_became_peered":"2024-04-25T00:00:02.527875+0000","last_unstale":"2024-04-25T00:00:55.654156+0000","last_undegraded":"2024-04-25T00:00:55.654156+0000","last_fullsized":"2024-04-25T00:00:55.654156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:06.634497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017250+0000","last_change":"2024-04-25T00:00:03.144246+0000","last_active":"2024-04-25T00:00:56.017250+0000","last_peered":"2024-04-25T00:00:56.017250+0000","last_clean":"2024-04-25T00:00:56.017250+0000","last_became_active":"2024-04-25T00:00:03.143925+0000","last_became_peered":"2024-04-25T00:00:03.143925+0000","last_unstale":"2024-04-25T00:00:56.017250+0000","last_undegraded":"2024-04-25T00:00:56.017250+0000","last_fullsized":"2024-04-25T00:00:56.017250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:51.741275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657464+0000","last_change":"2024-04-25T00:00:03.141261+0000","last_active":"2024-04-25T00:00:55.657464+0000","last_peered":"2024-04-25T00:00:55.657464+0000","last_clean":"2024-04-25T00:00:55.657464+0000","last_became_active":"2024-04-25T00:00:03.137943+0000","last_became_peered":"2024-04-25T00:00:03.137943+0000","last_unstale":"2024-04-25T00:00:55.657464+0000","last_undegraded":"2024-04-25T00:00:55.657464+0000","last_fullsized":"2024-04-25T00:00:55.657464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:41.655749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:15.676551+0000","last_change":"2024-04-25T00:00:02.522048+0000","last_active":"2024-04-25T00:01:15.676551+0000","last_peered":"2024-04-25T00:01:15.676551+0000","last_clean":"2024-04-25T00:01:15.676551+0000","last_became_active":"2024-04-25T00:00:02.521755+0000","last_became_peered":"2024-04-25T00:00:02.521755+0000","last_unstale":"2024-04-25T00:01:15.676551+0000","last_undegraded":"2024-04-25T00:01:15.676551+0000","last_fullsized":"2024-04-25T00:01:15.676551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:58.602357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656389+0000","last_change":"2024-04-25T00:00:03.151428+0000","last_active":"2024-04-25T00:00:55.656389+0000","last_peered":"2024-04-25T00:00:55.656389+0000","last_clean":"2024-04-25T00:00:55.656389+0000","last_became_active":"2024-04-25T00:00:03.151005+0000","last_became_peered":"2024-04-25T00:00:03.151005+0000","last_unstale":"2024-04-25T00:00:55.656389+0000","last_undegraded":"2024-04-25T00:00:55.656389+0000","last_fullsized":"2024-04-25T00:00:55.656389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:28.044803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653428+0000","last_change":"2024-04-25T00:00:02.515714+0000","last_active":"2024-04-25T00:00:55.653428+0000","last_peered":"2024-04-25T00:00:55.653428+0000","last_clean":"2024-04-25T00:00:55.653428+0000","last_became_active":"2024-04-25T00:00:02.515373+0000","last_became_peered":"2024-04-25T00:00:02.515373+0000","last_unstale":"2024-04-25T00:00:55.653428+0000","last_undegraded":"2024-04-25T00:00:55.653428+0000","last_fullsized":"2024-04-25T00:00:55.653428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:34.344261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654014+0000","last_change":"2024-04-24T23:58:50.595622+0000","last_active":"2024-04-25T00:00:55.654014+0000","last_peered":"2024-04-25T00:00:55.654014+0000","last_clean":"2024-04-25T00:00:55.654014+0000","last_became_active":"2024-04-24T23:58:50.594971+0000","last_became_peered":"2024-04-24T23:58:50.594971+0000","last_unstale":"2024-04-25T00:00:55.654014+0000","last_undegraded":"2024-04-25T00:00:55.654014+0000","last_fullsized":"2024-04-25T00:00:55.654014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:55:22.756734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:55:22.756734+0000","last_clean_scrub_stamp":"2024-04-24T23:55:22.756734+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:20.241007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653794+0000","last_change":"2024-04-25T00:00:03.135594+0000","last_active":"2024-04-25T00:00:55.653794+0000","last_peered":"2024-04-25T00:00:55.653794+0000","last_clean":"2024-04-25T00:00:55.653794+0000","last_became_active":"2024-04-25T00:00:03.135279+0000","last_became_peered":"2024-04-25T00:00:03.135279+0000","last_unstale":"2024-04-25T00:00:55.653794+0000","last_undegraded":"2024-04-25T00:00:55.653794+0000","last_fullsized":"2024-04-25T00:00:55.653794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:37.690119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.663160+0000","last_change":"2024-04-25T00:00:55.663160+0000","last_active":"2024-04-25T00:00:55.663160+0000","last_peered":"2024-04-25T00:00:55.663160+0000","last_clean":"2024-04-25T00:00:55.663160+0000","last_became_active":"2024-04-25T00:00:55.659082+0000","last_became_peered":"2024-04-25T00:00:55.659082+0000","last_unstale":"2024-04-25T00:00:55.663160+0000","last_undegraded":"2024-04-25T00:00:55.663160+0000","last_fullsized":"2024-04-25T00:00:55.663160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:20.586443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652407+0000","last_change":"2024-04-25T00:00:02.527545+0000","last_active":"2024-04-25T00:00:55.652407+0000","last_peered":"2024-04-25T00:00:55.652407+0000","last_clean":"2024-04-25T00:00:55.652407+0000","last_became_active":"2024-04-25T00:00:02.527250+0000","last_became_peered":"2024-04-25T00:00:02.527250+0000","last_unstale":"2024-04-25T00:00:55.652407+0000","last_undegraded":"2024-04-25T00:00:55.652407+0000","last_fullsized":"2024-04-25T00:00:55.652407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:57.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654599+0000","last_change":"2024-04-25T00:00:03.139578+0000","last_active":"2024-04-25T00:00:55.654599+0000","last_peered":"2024-04-25T00:00:55.654599+0000","last_clean":"2024-04-25T00:00:55.654599+0000","last_became_active":"2024-04-25T00:00:03.139128+0000","last_became_peered":"2024-04-25T00:00:03.139128+0000","last_unstale":"2024-04-25T00:00:55.654599+0000","last_undegraded":"2024-04-25T00:00:55.654599+0000","last_fullsized":"2024-04-25T00:00:55.654599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:33.509293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654125+0000","last_change":"2024-04-25T00:00:02.520616+0000","last_active":"2024-04-25T00:00:55.654125+0000","last_peered":"2024-04-25T00:00:55.654125+0000","last_clean":"2024-04-25T00:00:55.654125+0000","last_became_active":"2024-04-25T00:00:02.519057+0000","last_became_peered":"2024-04-25T00:00:02.519057+0000","last_unstale":"2024-04-25T00:00:55.654125+0000","last_undegraded":"2024-04-25T00:00:55.654125+0000","last_fullsized":"2024-04-25T00:00:55.654125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:25.847055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656655+0000","last_change":"2024-04-25T00:00:03.150010+0000","last_active":"2024-04-25T00:00:55.656655+0000","last_peered":"2024-04-25T00:00:55.656655+0000","last_clean":"2024-04-25T00:00:55.656655+0000","last_became_active":"2024-04-25T00:00:03.149677+0000","last_became_peered":"2024-04-25T00:00:03.149677+0000","last_unstale":"2024-04-25T00:00:55.656655+0000","last_undegraded":"2024-04-25T00:00:55.656655+0000","last_fullsized":"2024-04-25T00:00:55.656655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:18.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653112+0000","last_change":"2024-04-25T00:00:02.533171+0000","last_active":"2024-04-25T00:00:55.653112+0000","last_peered":"2024-04-25T00:00:55.653112+0000","last_clean":"2024-04-25T00:00:55.653112+0000","last_became_active":"2024-04-25T00:00:02.532887+0000","last_became_peered":"2024-04-25T00:00:02.532887+0000","last_unstale":"2024-04-25T00:00:55.653112+0000","last_undegraded":"2024-04-25T00:00:55.653112+0000","last_fullsized":"2024-04-25T00:00:55.653112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:24.729809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651467+0000","last_change":"2024-04-25T00:00:03.147457+0000","last_active":"2024-04-25T00:00:55.651467+0000","last_peered":"2024-04-25T00:00:55.651467+0000","last_clean":"2024-04-25T00:00:55.651467+0000","last_became_active":"2024-04-25T00:00:03.147163+0000","last_became_peered":"2024-04-25T00:00:03.147163+0000","last_unstale":"2024-04-25T00:00:55.651467+0000","last_undegraded":"2024-04-25T00:00:55.651467+0000","last_fullsized":"2024-04-25T00:00:55.651467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:32.422006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656376+0000","last_change":"2024-04-25T00:00:02.527573+0000","last_active":"2024-04-25T00:00:55.656376+0000","last_peered":"2024-04-25T00:00:55.656376+0000","last_clean":"2024-04-25T00:00:55.656376+0000","last_became_active":"2024-04-25T00:00:02.527267+0000","last_became_peered":"2024-04-25T00:00:02.527267+0000","last_unstale":"2024-04-25T00:00:55.656376+0000","last_undegraded":"2024-04-25T00:00:55.656376+0000","last_fullsized":"2024-04-25T00:00:55.656376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:31.581529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651843+0000","last_change":"2024-04-25T00:00:03.146602+0000","last_active":"2024-04-25T00:00:55.651843+0000","last_peered":"2024-04-25T00:00:55.651843+0000","last_clean":"2024-04-25T00:00:55.651843+0000","last_became_active":"2024-04-25T00:00:03.145480+0000","last_became_peered":"2024-04-25T00:00:03.145480+0000","last_unstale":"2024-04-25T00:00:55.651843+0000","last_undegraded":"2024-04-25T00:00:55.651843+0000","last_fullsized":"2024-04-25T00:00:55.651843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:02.803903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656153+0000","last_change":"2024-04-25T00:00:02.526567+0000","last_active":"2024-04-25T00:00:55.656153+0000","last_peered":"2024-04-25T00:00:55.656153+0000","last_clean":"2024-04-25T00:00:55.656153+0000","last_became_active":"2024-04-25T00:00:02.526234+0000","last_became_peered":"2024-04-25T00:00:02.526234+0000","last_unstale":"2024-04-25T00:00:55.656153+0000","last_undegraded":"2024-04-25T00:00:55.656153+0000","last_fullsized":"2024-04-25T00:00:55.656153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:14.964206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653520+0000","last_change":"2024-04-25T00:00:03.140959+0000","last_active":"2024-04-25T00:00:55.653520+0000","last_peered":"2024-04-25T00:00:55.653520+0000","last_clean":"2024-04-25T00:00:55.653520+0000","last_became_active":"2024-04-25T00:00:03.140533+0000","last_became_peered":"2024-04-25T00:00:03.140533+0000","last_unstale":"2024-04-25T00:00:55.653520+0000","last_undegraded":"2024-04-25T00:00:55.653520+0000","last_fullsized":"2024-04-25T00:00:55.653520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:46.166051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.664257+0000","last_change":"2024-04-25T00:00:55.663762+0000","last_active":"2024-04-25T00:00:55.664257+0000","last_peered":"2024-04-25T00:00:55.664257+0000","last_clean":"2024-04-25T00:00:55.664257+0000","last_became_active":"2024-04-25T00:00:55.657906+0000","last_became_peered":"2024-04-25T00:00:55.657906+0000","last_unstale":"2024-04-25T00:00:55.664257+0000","last_undegraded":"2024-04-25T00:00:55.664257+0000","last_fullsized":"2024-04-25T00:00:55.664257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:42.935465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656464+0000","last_change":"2024-04-25T00:00:03.149740+0000","last_active":"2024-04-25T00:00:55.656464+0000","last_peered":"2024-04-25T00:00:55.656464+0000","last_clean":"2024-04-25T00:00:55.656464+0000","last_became_active":"2024-04-25T00:00:03.149290+0000","last_became_peered":"2024-04-25T00:00:03.149290+0000","last_unstale":"2024-04-25T00:00:55.656464+0000","last_undegraded":"2024-04-25T00:00:55.656464+0000","last_fullsized":"2024-04-25T00:00:55.656464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.716974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652063+0000","last_change":"2024-04-25T00:00:02.526823+0000","last_active":"2024-04-25T00:00:55.652063+0000","last_peered":"2024-04-25T00:00:55.652063+0000","last_clean":"2024-04-25T00:00:55.652063+0000","last_became_active":"2024-04-25T00:00:02.526529+0000","last_became_peered":"2024-04-25T00:00:02.526529+0000","last_unstale":"2024-04-25T00:00:55.652063+0000","last_undegraded":"2024-04-25T00:00:55.652063+0000","last_fullsized":"2024-04-25T00:00:55.652063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:13.907985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654711+0000","last_change":"2024-04-25T00:00:03.141973+0000","last_active":"2024-04-25T00:00:55.654711+0000","last_peered":"2024-04-25T00:00:55.654711+0000","last_clean":"2024-04-25T00:00:55.654711+0000","last_became_active":"2024-04-25T00:00:03.141560+0000","last_became_peered":"2024-04-25T00:00:03.141560+0000","last_unstale":"2024-04-25T00:00:55.654711+0000","last_undegraded":"2024-04-25T00:00:55.654711+0000","last_fullsized":"2024-04-25T00:00:55.654711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:03.002939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657504+0000","last_change":"2024-04-25T00:00:02.533013+0000","last_active":"2024-04-25T00:00:55.657504+0000","last_peered":"2024-04-25T00:00:55.657504+0000","last_clean":"2024-04-25T00:00:55.657504+0000","last_became_active":"2024-04-25T00:00:02.532667+0000","last_became_peered":"2024-04-25T00:00:02.532667+0000","last_unstale":"2024-04-25T00:00:55.657504+0000","last_undegraded":"2024-04-25T00:00:55.657504+0000","last_fullsized":"2024-04-25T00:00:55.657504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:07.469136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654548+0000","last_change":"2024-04-25T00:00:03.140120+0000","last_active":"2024-04-25T00:00:55.654548+0000","last_peered":"2024-04-25T00:00:55.654548+0000","last_clean":"2024-04-25T00:00:55.654548+0000","last_became_active":"2024-04-25T00:00:03.139778+0000","last_became_peered":"2024-04-25T00:00:03.139778+0000","last_unstale":"2024-04-25T00:00:55.654548+0000","last_undegraded":"2024-04-25T00:00:55.654548+0000","last_fullsized":"2024-04-25T00:00:55.654548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:02.678303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657422+0000","last_change":"2024-04-25T00:00:02.534359+0000","last_active":"2024-04-25T00:00:55.657422+0000","last_peered":"2024-04-25T00:00:55.657422+0000","last_clean":"2024-04-25T00:00:55.657422+0000","last_became_active":"2024-04-25T00:00:02.534057+0000","last_became_peered":"2024-04-25T00:00:02.534057+0000","last_unstale":"2024-04-25T00:00:55.657422+0000","last_undegraded":"2024-04-25T00:00:55.657422+0000","last_fullsized":"2024-04-25T00:00:55.657422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:38.716263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654205+0000","last_change":"2024-04-25T00:00:03.146608+0000","last_active":"2024-04-25T00:00:55.654205+0000","last_peered":"2024-04-25T00:00:55.654205+0000","last_clean":"2024-04-25T00:00:55.654205+0000","last_became_active":"2024-04-25T00:00:03.146256+0000","last_became_peered":"2024-04-25T00:00:03.146256+0000","last_unstale":"2024-04-25T00:00:55.654205+0000","last_undegraded":"2024-04-25T00:00:55.654205+0000","last_fullsized":"2024-04-25T00:00:55.654205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:04.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.661162+0000","last_change":"2024-04-25T00:00:55.661162+0000","last_active":"2024-04-25T00:00:55.661162+0000","last_peered":"2024-04-25T00:00:55.661162+0000","last_clean":"2024-04-25T00:00:55.661162+0000","last_became_active":"2024-04-25T00:00:55.657844+0000","last_became_peered":"2024-04-25T00:00:55.657844+0000","last_unstale":"2024-04-25T00:00:55.661162+0000","last_undegraded":"2024-04-25T00:00:55.661162+0000","last_fullsized":"2024-04-25T00:00:55.661162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:18.458244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654438+0000","last_change":"2024-04-25T00:00:03.141296+0000","last_active":"2024-04-25T00:00:55.654438+0000","last_peered":"2024-04-25T00:00:55.654438+0000","last_clean":"2024-04-25T00:00:55.654438+0000","last_became_active":"2024-04-25T00:00:03.139916+0000","last_became_peered":"2024-04-25T00:00:03.139916+0000","last_unstale":"2024-04-25T00:00:55.654438+0000","last_undegraded":"2024-04-25T00:00:55.654438+0000","last_fullsized":"2024-04-25T00:00:55.654438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:24.534210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653635+0000","last_change":"2024-04-25T00:00:03.149427+0000","last_active":"2024-04-25T00:00:55.653635+0000","last_peered":"2024-04-25T00:00:55.653635+0000","last_clean":"2024-04-25T00:00:55.653635+0000","last_became_active":"2024-04-25T00:00:03.148445+0000","last_became_peered":"2024-04-25T00:00:03.148445+0000","last_unstale":"2024-04-25T00:00:55.653635+0000","last_undegraded":"2024-04-25T00:00:55.653635+0000","last_fullsized":"2024-04-25T00:00:55.653635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:35.212153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657285+0000","last_change":"2024-04-25T00:00:03.146994+0000","last_active":"2024-04-25T00:00:55.657285+0000","last_peered":"2024-04-25T00:00:55.657285+0000","last_clean":"2024-04-25T00:00:55.657285+0000","last_became_active":"2024-04-25T00:00:03.146677+0000","last_became_peered":"2024-04-25T00:00:03.146677+0000","last_unstale":"2024-04-25T00:00:55.657285+0000","last_undegraded":"2024-04-25T00:00:55.657285+0000","last_fullsized":"2024-04-25T00:00:55.657285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:33.883583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656381+0000","last_change":"2024-04-25T00:00:03.149428+0000","last_active":"2024-04-25T00:00:55.656381+0000","last_peered":"2024-04-25T00:00:55.656381+0000","last_clean":"2024-04-25T00:00:55.656381+0000","last_became_active":"2024-04-25T00:00:03.147978+0000","last_became_peered":"2024-04-25T00:00:03.147978+0000","last_unstale":"2024-04-25T00:00:55.656381+0000","last_undegraded":"2024-04-25T00:00:55.656381+0000","last_fullsized":"2024-04-25T00:00:55.656381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:34.684463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651336+0000","last_change":"2024-04-25T00:00:03.147528+0000","last_active":"2024-04-25T00:00:55.651336+0000","last_peered":"2024-04-25T00:00:55.651336+0000","last_clean":"2024-04-25T00:00:55.651336+0000","last_became_active":"2024-04-25T00:00:03.147182+0000","last_became_peered":"2024-04-25T00:00:03.147182+0000","last_unstale":"2024-04-25T00:00:55.651336+0000","last_undegraded":"2024-04-25T00:00:55.651336+0000","last_fullsized":"2024-04-25T00:00:55.651336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:27.592335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654696+0000","last_change":"2024-04-25T00:00:03.143274+0000","last_active":"2024-04-25T00:00:55.654696+0000","last_peered":"2024-04-25T00:00:55.654696+0000","last_clean":"2024-04-25T00:00:55.654696+0000","last_became_active":"2024-04-25T00:00:03.142384+0000","last_became_peered":"2024-04-25T00:00:03.142384+0000","last_unstale":"2024-04-25T00:00:55.654696+0000","last_undegraded":"2024-04-25T00:00:55.654696+0000","last_fullsized":"2024-04-25T00:00:55.654696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:41.118123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653696+0000","last_change":"2024-04-25T00:00:03.147769+0000","last_active":"2024-04-25T00:00:55.653696+0000","last_peered":"2024-04-25T00:00:55.653696+0000","last_clean":"2024-04-25T00:00:55.653696+0000","last_became_active":"2024-04-25T00:00:03.147009+0000","last_became_peered":"2024-04-25T00:00:03.147009+0000","last_unstale":"2024-04-25T00:00:55.653696+0000","last_undegraded":"2024-04-25T00:00:55.653696+0000","last_fullsized":"2024-04-25T00:00:55.653696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:18.917532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.658543+0000","last_change":"2024-04-25T00:00:03.141519+0000","last_active":"2024-04-25T00:00:55.658543+0000","last_peered":"2024-04-25T00:00:55.658543+0000","last_clean":"2024-04-25T00:00:55.658543+0000","last_became_active":"2024-04-25T00:00:03.139391+0000","last_became_peered":"2024-04-25T00:00:03.139391+0000","last_unstale":"2024-04-25T00:00:55.658543+0000","last_undegraded":"2024-04-25T00:00:55.658543+0000","last_fullsized":"2024-04-25T00:00:55.658543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:13.043899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656618+0000","last_change":"2024-04-25T00:00:03.151311+0000","last_active":"2024-04-25T00:00:55.656618+0000","last_peered":"2024-04-25T00:00:55.656618+0000","last_clean":"2024-04-25T00:00:55.656618+0000","last_became_active":"2024-04-25T00:00:03.150992+0000","last_became_peered":"2024-04-25T00:00:03.150992+0000","last_unstale":"2024-04-25T00:00:55.656618+0000","last_undegraded":"2024-04-25T00:00:55.656618+0000","last_fullsized":"2024-04-25T00:00:55.656618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:47.572641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657364+0000","last_change":"2024-04-25T00:00:03.145623+0000","last_active":"2024-04-25T00:00:55.657364+0000","last_peered":"2024-04-25T00:00:55.657364+0000","last_clean":"2024-04-25T00:00:55.657364+0000","last_became_active":"2024-04-25T00:00:03.145260+0000","last_became_peered":"2024-04-25T00:00:03.145260+0000","last_unstale":"2024-04-25T00:00:55.657364+0000","last_undegraded":"2024-04-25T00:00:55.657364+0000","last_fullsized":"2024-04-25T00:00:55.657364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:16.985732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656105+0000","last_change":"2024-04-25T00:00:03.144957+0000","last_active":"2024-04-25T00:00:55.656105+0000","last_peered":"2024-04-25T00:00:55.656105+0000","last_clean":"2024-04-25T00:00:55.656105+0000","last_became_active":"2024-04-25T00:00:03.144631+0000","last_became_peered":"2024-04-25T00:00:03.144631+0000","last_unstale":"2024-04-25T00:00:55.656105+0000","last_undegraded":"2024-04-25T00:00:55.656105+0000","last_fullsized":"2024-04-25T00:00:55.656105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.636553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656487+0000","last_change":"2024-04-25T00:00:03.146574+0000","last_active":"2024-04-25T00:00:55.656487+0000","last_peered":"2024-04-25T00:00:55.656487+0000","last_clean":"2024-04-25T00:00:55.656487+0000","last_became_active":"2024-04-25T00:00:03.145095+0000","last_became_peered":"2024-04-25T00:00:03.145095+0000","last_unstale":"2024-04-25T00:00:55.656487+0000","last_undegraded":"2024-04-25T00:00:55.656487+0000","last_fullsized":"2024-04-25T00:00:55.656487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:38.060170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654352+0000","last_change":"2024-04-25T00:00:03.143958+0000","last_active":"2024-04-25T00:00:55.654352+0000","last_peered":"2024-04-25T00:00:55.654352+0000","last_clean":"2024-04-25T00:00:55.654352+0000","last_became_active":"2024-04-25T00:00:03.143621+0000","last_became_peered":"2024-04-25T00:00:03.143621+0000","last_unstale":"2024-04-25T00:00:55.654352+0000","last_undegraded":"2024-04-25T00:00:55.654352+0000","last_fullsized":"2024-04-25T00:00:55.654352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:21.833252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654417+0000","last_change":"2024-04-25T00:00:03.143703+0000","last_active":"2024-04-25T00:00:55.654417+0000","last_peered":"2024-04-25T00:00:55.654417+0000","last_clean":"2024-04-25T00:00:55.654417+0000","last_became_active":"2024-04-25T00:00:03.143400+0000","last_became_peered":"2024-04-25T00:00:03.143400+0000","last_unstale":"2024-04-25T00:00:55.654417+0000","last_undegraded":"2024-04-25T00:00:55.654417+0000","last_fullsized":"2024-04-25T00:00:55.654417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:53.984769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284249,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63158,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":458,"num_read_kb":290,"num_write":157,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":189474,"data_compressed":5559,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":456102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":1106740,"data_compressed":37998,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1048579,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":455727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":514286,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":456998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":516027,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":456038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":456555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":1047872,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58766,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60507,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60735,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:23.533 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:01:23.780 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:23.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:23 smithi110 ceph-mon[29187]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:23 smithi188 ceph-mon[33285]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:23 smithi173 ceph-mon[32402]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:24.987 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:24 smithi110 ceph-mon[29187]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:01:24.987 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:24 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:24.987 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:24 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:24 smithi188 ceph-mon[33285]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:01:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:24 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:24 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:25.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:24 smithi173 ceph-mon[32402]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:01:25.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:24 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:25.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:24 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:25.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 -- 172.21.15.110:0/2077808855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea44107570 msgr2=0x7fea4410f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 --2- 172.21.15.110:0/2077808855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea44107570 0x7fea4410f140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009990 tx=0x7fea3c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 -- 172.21.15.110:0/2077808855 shutdown_connections 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 --2- 172.21.15.110:0/2077808855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea44107570 0x7fea4410f140 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 --2- 172.21.15.110:0/2077808855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea441024e0 0x7fea441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 --2- 172.21.15.110:0/2077808855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea44102e90 0x7fea44106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 -- 172.21.15.110:0/2077808855 >> 172.21.15.110:0/2077808855 conn(0x7fea44079390 msgr2=0x7fea440797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 -- 172.21.15.110:0/2077808855 shutdown_connections 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 -- 172.21.15.110:0/2077808855 wait complete. 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.541+0000 7fea4a13c700 1 Processor -- start 2024-04-25T00:01:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea4a13c700 1 -- start start 2024-04-25T00:01:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea4a13c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441024e0 0x7fea4419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea4a13c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea44102e90 0x7fea4419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea4a13c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea44107570 0x7fea44198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea4a13c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea44112460 con 0x7fea441024e0 2024-04-25T00:01:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea4a13c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea441122e0 con 0x7fea44102e90 2024-04-25T00:01:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea4a13c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea441125e0 con 0x7fea44107570 2024-04-25T00:01:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441024e0 0x7fea4419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441024e0 0x7fea4419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51004/0 (socket says 172.21.15.110:51004) 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 -- 172.21.15.110:0/16939879 learned_addr learned my addr 172.21.15.110:0/16939879 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea43fff700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea44107570 0x7fea44198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea42ffd700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea44102e90 0x7fea4419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 -- 172.21.15.110:0/16939879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea44107570 msgr2=0x7fea44198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea44107570 0x7fea44198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 -- 172.21.15.110:0/16939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea44102e90 msgr2=0x7fea4419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea44102e90 0x7fea4419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 -- 172.21.15.110:0/16939879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea44198f60 con 0x7fea441024e0 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea43fff700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea44107570 0x7fea44198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea42ffd700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea44102e90 0x7fea4419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.542+0000 7fea437fe700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441024e0 0x7fea4419e600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea3400ada0 tx=0x7fea3400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.543+0000 7fea40ff9700 1 -- 172.21.15.110:0/16939879 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34017020 con 0x7fea441024e0 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.543+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea44199250 con 0x7fea441024e0 2024-04-25T00:01:25.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.543+0000 7fea40ff9700 1 -- 172.21.15.110:0/16939879 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea3400ce30 con 0x7fea441024e0 2024-04-25T00:01:25.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.543+0000 7fea40ff9700 1 -- 172.21.15.110:0/16939879 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34005560 con 0x7fea441024e0 2024-04-25T00:01:25.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.543+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea441a5460 con 0x7fea441024e0 2024-04-25T00:01:25.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.544+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea44103970 con 0x7fea441024e0 2024-04-25T00:01:25.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.545+0000 7fea40ff9700 1 -- 172.21.15.110:0/16939879 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fea34004ad0 con 0x7fea441024e0 2024-04-25T00:01:25.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.545+0000 7fea40ff9700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fea300a2c20 0x7fea300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:25.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.545+0000 7fea42ffd700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fea300a2c20 0x7fea300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:25.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.545+0000 7fea40ff9700 1 -- 172.21.15.110:0/16939879 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fea340ce920 con 0x7fea441024e0 2024-04-25T00:01:25.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.546+0000 7fea42ffd700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fea300a2c20 0x7fea300a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea44108aa0 tx=0x7fea2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:25.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.550+0000 7fea40ff9700 1 -- 172.21.15.110:0/16939879 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fea34021020 con 0x7fea441024e0 2024-04-25T00:01:25.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.746+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea44062670 con 0x7fea300a2c20 2024-04-25T00:01:25.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.753+0000 7fea40ff9700 1 -- 172.21.15.110:0/16939879 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258930 (secure 0 0 0) 0x7fea44062670 con 0x7fea300a2c20 2024-04-25T00:01:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fea300a2c20 msgr2=0x7fea300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fea300a2c20 0x7fea300a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea44108aa0 tx=0x7fea2c008040 comp rx=0 tx=0).stop 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441024e0 msgr2=0x7fea4419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441024e0 0x7fea4419e600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea3400ada0 tx=0x7fea3400c330 comp rx=0 tx=0).stop 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 shutdown_connections 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fea300a2c20 0x7fea300a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea44107570 0x7fea44198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea441024e0 0x7fea4419e600 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 --2- 172.21.15.110:0/16939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea44102e90 0x7fea4419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:25.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 >> 172.21.15.110:0/16939879 conn(0x7fea44079390 msgr2=0x7fea44101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 shutdown_connections 2024-04-25T00:01:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:25.756+0000 7fea4a13c700 1 -- 172.21.15.110:0/16939879 wait complete. 2024-04-25T00:01:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-04-25T00:01:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:25 smithi188 ceph-mon[33285]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:25 smithi188 ceph-mon[33285]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:01:26.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:01:26.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:01:26.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:25 smithi110 ceph-mon[29187]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:25 smithi110 ceph-mon[29187]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:01:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:01:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:01:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:25 smithi173 ceph-mon[32402]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:25 smithi173 ceph-mon[32402]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:01:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:01:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:01:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:01:26.511 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-25T00:01:24.528357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653760,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":604,"num_read_kb":414,"num_write":340,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352468,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589356,"statfs":{"total":1151940427776,"available":1151579500544,"internally_reserved":0,"allocated":11833344,"data_stored":7427520,"data_compressed":113994,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002904"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654019+0000","last_change":"2024-04-25T00:00:03.143150+0000","last_active":"2024-04-25T00:00:55.654019+0000","last_peered":"2024-04-25T00:00:55.654019+0000","last_clean":"2024-04-25T00:00:55.654019+0000","last_became_active":"2024-04-25T00:00:03.142344+0000","last_became_peered":"2024-04-25T00:00:03.142344+0000","last_unstale":"2024-04-25T00:00:55.654019+0000","last_undegraded":"2024-04-25T00:00:55.654019+0000","last_fullsized":"2024-04-25T00:00:55.654019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:17:18.293748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651560+0000","last_change":"2024-04-25T00:00:03.145454+0000","last_active":"2024-04-25T00:00:55.651560+0000","last_peered":"2024-04-25T00:00:55.651560+0000","last_clean":"2024-04-25T00:00:55.651560+0000","last_became_active":"2024-04-25T00:00:03.144905+0000","last_became_peered":"2024-04-25T00:00:03.144905+0000","last_unstale":"2024-04-25T00:00:55.651560+0000","last_undegraded":"2024-04-25T00:00:55.651560+0000","last_fullsized":"2024-04-25T00:00:55.651560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:39.583313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651373+0000","last_change":"2024-04-25T00:00:03.141770+0000","last_active":"2024-04-25T00:00:55.651373+0000","last_peered":"2024-04-25T00:00:55.651373+0000","last_clean":"2024-04-25T00:00:55.651373+0000","last_became_active":"2024-04-25T00:00:03.141468+0000","last_became_peered":"2024-04-25T00:00:03.141468+0000","last_unstale":"2024-04-25T00:00:55.651373+0000","last_undegraded":"2024-04-25T00:00:55.651373+0000","last_fullsized":"2024-04-25T00:00:55.651373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:33.112112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.660753+0000","last_change":"2024-04-25T00:00:55.660277+0000","last_active":"2024-04-25T00:00:55.660753+0000","last_peered":"2024-04-25T00:00:55.660753+0000","last_clean":"2024-04-25T00:00:55.660753+0000","last_became_active":"2024-04-25T00:00:55.659851+0000","last_became_peered":"2024-04-25T00:00:55.659851+0000","last_unstale":"2024-04-25T00:00:55.660753+0000","last_undegraded":"2024-04-25T00:00:55.660753+0000","last_fullsized":"2024-04-25T00:00:55.660753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:04.471838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652056+0000","last_change":"2024-04-25T00:00:03.146405+0000","last_active":"2024-04-25T00:00:55.652056+0000","last_peered":"2024-04-25T00:00:55.652056+0000","last_clean":"2024-04-25T00:00:55.652056+0000","last_became_active":"2024-04-25T00:00:03.145520+0000","last_became_peered":"2024-04-25T00:00:03.145520+0000","last_unstale":"2024-04-25T00:00:55.652056+0000","last_undegraded":"2024-04-25T00:00:55.652056+0000","last_fullsized":"2024-04-25T00:00:55.652056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:02.256905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017163+0000","last_change":"2024-04-25T00:00:03.142973+0000","last_active":"2024-04-25T00:00:56.017163+0000","last_peered":"2024-04-25T00:00:56.017163+0000","last_clean":"2024-04-25T00:00:56.017163+0000","last_became_active":"2024-04-25T00:00:03.142091+0000","last_became_peered":"2024-04-25T00:00:03.142091+0000","last_unstale":"2024-04-25T00:00:56.017163+0000","last_undegraded":"2024-04-25T00:00:56.017163+0000","last_fullsized":"2024-04-25T00:00:56.017163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:01.948299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653849+0000","last_change":"2024-04-25T00:00:03.148383+0000","last_active":"2024-04-25T00:00:55.653849+0000","last_peered":"2024-04-25T00:00:55.653849+0000","last_clean":"2024-04-25T00:00:55.653849+0000","last_became_active":"2024-04-25T00:00:03.148105+0000","last_became_peered":"2024-04-25T00:00:03.148105+0000","last_unstale":"2024-04-25T00:00:55.653849+0000","last_undegraded":"2024-04-25T00:00:55.653849+0000","last_fullsized":"2024-04-25T00:00:55.653849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:31.902600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656094+0000","last_change":"2024-04-25T00:00:03.151575+0000","last_active":"2024-04-25T00:00:55.656094+0000","last_peered":"2024-04-25T00:00:55.656094+0000","last_clean":"2024-04-25T00:00:55.656094+0000","last_became_active":"2024-04-25T00:00:03.151269+0000","last_became_peered":"2024-04-25T00:00:03.151269+0000","last_unstale":"2024-04-25T00:00:55.656094+0000","last_undegraded":"2024-04-25T00:00:55.656094+0000","last_fullsized":"2024-04-25T00:00:55.656094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:16.561180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657083+0000","last_change":"2024-04-25T00:00:03.140175+0000","last_active":"2024-04-25T00:00:55.657083+0000","last_peered":"2024-04-25T00:00:55.657083+0000","last_clean":"2024-04-25T00:00:55.657083+0000","last_became_active":"2024-04-25T00:00:03.139838+0000","last_became_peered":"2024-04-25T00:00:03.139838+0000","last_unstale":"2024-04-25T00:00:55.657083+0000","last_undegraded":"2024-04-25T00:00:55.657083+0000","last_fullsized":"2024-04-25T00:00:55.657083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:16.768025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017489+0000","last_change":"2024-04-25T00:00:03.143149+0000","last_active":"2024-04-25T00:00:56.017489+0000","last_peered":"2024-04-25T00:00:56.017489+0000","last_clean":"2024-04-25T00:00:56.017489+0000","last_became_active":"2024-04-25T00:00:03.142799+0000","last_became_peered":"2024-04-25T00:00:03.142799+0000","last_unstale":"2024-04-25T00:00:56.017489+0000","last_undegraded":"2024-04-25T00:00:56.017489+0000","last_fullsized":"2024-04-25T00:00:56.017489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:26:38.941329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654510+0000","last_change":"2024-04-25T00:00:03.141519+0000","last_active":"2024-04-25T00:00:55.654510+0000","last_peered":"2024-04-25T00:00:55.654510+0000","last_clean":"2024-04-25T00:00:55.654510+0000","last_became_active":"2024-04-25T00:00:03.140452+0000","last_became_peered":"2024-04-25T00:00:03.140452+0000","last_unstale":"2024-04-25T00:00:55.654510+0000","last_undegraded":"2024-04-25T00:00:55.654510+0000","last_fullsized":"2024-04-25T00:00:55.654510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:17.552403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653419+0000","last_change":"2024-04-25T00:00:03.149727+0000","last_active":"2024-04-25T00:00:55.653419+0000","last_peered":"2024-04-25T00:00:55.653419+0000","last_clean":"2024-04-25T00:00:55.653419+0000","last_became_active":"2024-04-25T00:00:03.149205+0000","last_became_peered":"2024-04-25T00:00:03.149205+0000","last_unstale":"2024-04-25T00:00:55.653419+0000","last_undegraded":"2024-04-25T00:00:55.653419+0000","last_fullsized":"2024-04-25T00:00:55.653419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:55.782750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654313+0000","last_change":"2024-04-25T00:00:03.139219+0000","last_active":"2024-04-25T00:00:55.654313+0000","last_peered":"2024-04-25T00:00:55.654313+0000","last_clean":"2024-04-25T00:00:55.654313+0000","last_became_active":"2024-04-25T00:00:03.138883+0000","last_became_peered":"2024-04-25T00:00:03.138883+0000","last_unstale":"2024-04-25T00:00:55.654313+0000","last_undegraded":"2024-04-25T00:00:55.654313+0000","last_fullsized":"2024-04-25T00:00:55.654313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.744932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651272+0000","last_change":"2024-04-25T00:00:03.144473+0000","last_active":"2024-04-25T00:00:55.651272+0000","last_peered":"2024-04-25T00:00:55.651272+0000","last_clean":"2024-04-25T00:00:55.651272+0000","last_became_active":"2024-04-25T00:00:03.144166+0000","last_became_peered":"2024-04-25T00:00:03.144166+0000","last_unstale":"2024-04-25T00:00:55.651272+0000","last_undegraded":"2024-04-25T00:00:55.651272+0000","last_fullsized":"2024-04-25T00:00:55.651272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:25.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651170+0000","last_change":"2024-04-25T00:00:03.143292+0000","last_active":"2024-04-25T00:00:55.651170+0000","last_peered":"2024-04-25T00:00:55.651170+0000","last_clean":"2024-04-25T00:00:55.651170+0000","last_became_active":"2024-04-25T00:00:03.142993+0000","last_became_peered":"2024-04-25T00:00:03.142993+0000","last_unstale":"2024-04-25T00:00:55.651170+0000","last_undegraded":"2024-04-25T00:00:55.651170+0000","last_fullsized":"2024-04-25T00:00:55.651170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.841015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656227+0000","last_change":"2024-04-25T00:00:03.148117+0000","last_active":"2024-04-25T00:00:55.656227+0000","last_peered":"2024-04-25T00:00:55.656227+0000","last_clean":"2024-04-25T00:00:55.656227+0000","last_became_active":"2024-04-25T00:00:03.147670+0000","last_became_peered":"2024-04-25T00:00:03.147670+0000","last_unstale":"2024-04-25T00:00:55.656227+0000","last_undegraded":"2024-04-25T00:00:55.656227+0000","last_fullsized":"2024-04-25T00:00:55.656227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:01.716253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654118+0000","last_change":"2024-04-25T00:00:03.143452+0000","last_active":"2024-04-25T00:00:55.654118+0000","last_peered":"2024-04-25T00:00:55.654118+0000","last_clean":"2024-04-25T00:00:55.654118+0000","last_became_active":"2024-04-25T00:00:03.142379+0000","last_became_peered":"2024-04-25T00:00:03.142379+0000","last_unstale":"2024-04-25T00:00:55.654118+0000","last_undegraded":"2024-04-25T00:00:55.654118+0000","last_fullsized":"2024-04-25T00:00:55.654118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:46.828363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654235+0000","last_change":"2024-04-25T00:00:03.138570+0000","last_active":"2024-04-25T00:00:55.654235+0000","last_peered":"2024-04-25T00:00:55.654235+0000","last_clean":"2024-04-25T00:00:55.654235+0000","last_became_active":"2024-04-25T00:00:03.138198+0000","last_became_peered":"2024-04-25T00:00:03.138198+0000","last_unstale":"2024-04-25T00:00:55.654235+0000","last_undegraded":"2024-04-25T00:00:55.654235+0000","last_fullsized":"2024-04-25T00:00:55.654235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:47.794135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656684+0000","last_change":"2024-04-25T00:00:03.143152+0000","last_active":"2024-04-25T00:00:55.656684+0000","last_peered":"2024-04-25T00:00:55.656684+0000","last_clean":"2024-04-25T00:00:55.656684+0000","last_became_active":"2024-04-25T00:00:03.142812+0000","last_became_peered":"2024-04-25T00:00:03.142812+0000","last_unstale":"2024-04-25T00:00:55.656684+0000","last_undegraded":"2024-04-25T00:00:55.656684+0000","last_fullsized":"2024-04-25T00:00:55.656684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:58.808783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654008+0000","last_change":"2024-04-25T00:00:02.525740+0000","last_active":"2024-04-25T00:00:55.654008+0000","last_peered":"2024-04-25T00:00:55.654008+0000","last_clean":"2024-04-25T00:00:55.654008+0000","last_became_active":"2024-04-25T00:00:02.525429+0000","last_became_peered":"2024-04-25T00:00:02.525429+0000","last_unstale":"2024-04-25T00:00:55.654008+0000","last_undegraded":"2024-04-25T00:00:55.654008+0000","last_fullsized":"2024-04-25T00:00:55.654008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:43.585678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.660547+0000","last_change":"2024-04-25T00:00:55.660072+0000","last_active":"2024-04-25T00:00:55.660547+0000","last_peered":"2024-04-25T00:00:55.660547+0000","last_clean":"2024-04-25T00:00:55.660547+0000","last_became_active":"2024-04-25T00:00:55.659804+0000","last_became_peered":"2024-04-25T00:00:55.659804+0000","last_unstale":"2024-04-25T00:00:55.660547+0000","last_undegraded":"2024-04-25T00:00:55.660547+0000","last_fullsized":"2024-04-25T00:00:55.660547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:52.780468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656749+0000","last_change":"2024-04-25T00:00:02.534181+0000","last_active":"2024-04-25T00:00:55.656749+0000","last_peered":"2024-04-25T00:00:55.656749+0000","last_clean":"2024-04-25T00:00:55.656749+0000","last_became_active":"2024-04-25T00:00:02.533848+0000","last_became_peered":"2024-04-25T00:00:02.533848+0000","last_unstale":"2024-04-25T00:00:55.656749+0000","last_undegraded":"2024-04-25T00:00:55.656749+0000","last_fullsized":"2024-04-25T00:00:55.656749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:25.276477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651607+0000","last_change":"2024-04-25T00:00:03.142038+0000","last_active":"2024-04-25T00:00:55.651607+0000","last_peered":"2024-04-25T00:00:55.651607+0000","last_clean":"2024-04-25T00:00:55.651607+0000","last_became_active":"2024-04-25T00:00:03.141705+0000","last_became_peered":"2024-04-25T00:00:03.141705+0000","last_unstale":"2024-04-25T00:00:55.651607+0000","last_undegraded":"2024-04-25T00:00:55.651607+0000","last_fullsized":"2024-04-25T00:00:55.651607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:58.382031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652506+0000","last_change":"2024-04-25T00:00:02.526339+0000","last_active":"2024-04-25T00:00:55.652506+0000","last_peered":"2024-04-25T00:00:55.652506+0000","last_clean":"2024-04-25T00:00:55.652506+0000","last_became_active":"2024-04-25T00:00:02.525999+0000","last_became_peered":"2024-04-25T00:00:02.525999+0000","last_unstale":"2024-04-25T00:00:55.652506+0000","last_undegraded":"2024-04-25T00:00:55.652506+0000","last_fullsized":"2024-04-25T00:00:55.652506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:29:52.997821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656619+0000","last_change":"2024-04-25T00:00:03.143896+0000","last_active":"2024-04-25T00:00:55.656619+0000","last_peered":"2024-04-25T00:00:55.656619+0000","last_clean":"2024-04-25T00:00:55.656619+0000","last_became_active":"2024-04-25T00:00:03.141311+0000","last_became_peered":"2024-04-25T00:00:03.141311+0000","last_unstale":"2024-04-25T00:00:55.656619+0000","last_undegraded":"2024-04-25T00:00:55.656619+0000","last_fullsized":"2024-04-25T00:00:55.656619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:57.819319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.666658+0000","last_change":"2024-04-25T00:00:55.666220+0000","last_active":"2024-04-25T00:00:55.666658+0000","last_peered":"2024-04-25T00:00:55.666658+0000","last_clean":"2024-04-25T00:00:55.666658+0000","last_became_active":"2024-04-25T00:00:55.659538+0000","last_became_peered":"2024-04-25T00:00:55.659538+0000","last_unstale":"2024-04-25T00:00:55.666658+0000","last_undegraded":"2024-04-25T00:00:55.666658+0000","last_fullsized":"2024-04-25T00:00:55.666658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:05.888688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657973+0000","last_change":"2024-04-25T00:00:03.140867+0000","last_active":"2024-04-25T00:00:55.657973+0000","last_peered":"2024-04-25T00:00:55.657973+0000","last_clean":"2024-04-25T00:00:55.657973+0000","last_became_active":"2024-04-25T00:00:03.140545+0000","last_became_peered":"2024-04-25T00:00:03.140545+0000","last_unstale":"2024-04-25T00:00:55.657973+0000","last_undegraded":"2024-04-25T00:00:55.657973+0000","last_fullsized":"2024-04-25T00:00:55.657973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:46.928359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:20.668943+0000","last_change":"2024-04-25T00:00:02.528897+0000","last_active":"2024-04-25T00:01:20.668943+0000","last_peered":"2024-04-25T00:01:20.668943+0000","last_clean":"2024-04-25T00:01:20.668943+0000","last_became_active":"2024-04-25T00:00:02.528584+0000","last_became_peered":"2024-04-25T00:00:02.528584+0000","last_unstale":"2024-04-25T00:01:20.668943+0000","last_undegraded":"2024-04-25T00:01:20.668943+0000","last_fullsized":"2024-04-25T00:01:20.668943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:23.437956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651937+0000","last_change":"2024-04-25T00:00:03.147692+0000","last_active":"2024-04-25T00:00:55.651937+0000","last_peered":"2024-04-25T00:00:55.651937+0000","last_clean":"2024-04-25T00:00:55.651937+0000","last_became_active":"2024-04-25T00:00:03.147183+0000","last_became_peered":"2024-04-25T00:00:03.147183+0000","last_unstale":"2024-04-25T00:00:55.651937+0000","last_undegraded":"2024-04-25T00:00:55.651937+0000","last_fullsized":"2024-04-25T00:00:55.651937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:12.640665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017105+0000","last_change":"2024-04-25T00:00:02.522387+0000","last_active":"2024-04-25T00:00:56.017105+0000","last_peered":"2024-04-25T00:00:56.017105+0000","last_clean":"2024-04-25T00:00:56.017105+0000","last_became_active":"2024-04-25T00:00:02.522093+0000","last_became_peered":"2024-04-25T00:00:02.522093+0000","last_unstale":"2024-04-25T00:00:56.017105+0000","last_undegraded":"2024-04-25T00:00:56.017105+0000","last_fullsized":"2024-04-25T00:00:56.017105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:33.393915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657355+0000","last_change":"2024-04-25T00:00:03.146277+0000","last_active":"2024-04-25T00:00:55.657355+0000","last_peered":"2024-04-25T00:00:55.657355+0000","last_clean":"2024-04-25T00:00:55.657355+0000","last_became_active":"2024-04-25T00:00:03.145846+0000","last_became_peered":"2024-04-25T00:00:03.145846+0000","last_unstale":"2024-04-25T00:00:55.657355+0000","last_undegraded":"2024-04-25T00:00:55.657355+0000","last_fullsized":"2024-04-25T00:00:55.657355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.692246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654117+0000","last_change":"2024-04-25T00:00:02.528405+0000","last_active":"2024-04-25T00:00:55.654117+0000","last_peered":"2024-04-25T00:00:55.654117+0000","last_clean":"2024-04-25T00:00:55.654117+0000","last_became_active":"2024-04-25T00:00:02.528115+0000","last_became_peered":"2024-04-25T00:00:02.528115+0000","last_unstale":"2024-04-25T00:00:55.654117+0000","last_undegraded":"2024-04-25T00:00:55.654117+0000","last_fullsized":"2024-04-25T00:00:55.654117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:42:51.382361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.655982+0000","last_change":"2024-04-25T00:00:03.144266+0000","last_active":"2024-04-25T00:00:55.655982+0000","last_peered":"2024-04-25T00:00:55.655982+0000","last_clean":"2024-04-25T00:00:55.655982+0000","last_became_active":"2024-04-25T00:00:03.141704+0000","last_became_peered":"2024-04-25T00:00:03.141704+0000","last_unstale":"2024-04-25T00:00:55.655982+0000","last_undegraded":"2024-04-25T00:00:55.655982+0000","last_fullsized":"2024-04-25T00:00:55.655982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:14.161783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657192+0000","last_change":"2024-04-25T00:00:02.528134+0000","last_active":"2024-04-25T00:00:55.657192+0000","last_peered":"2024-04-25T00:00:55.657192+0000","last_clean":"2024-04-25T00:00:55.657192+0000","last_became_active":"2024-04-25T00:00:02.527830+0000","last_became_peered":"2024-04-25T00:00:02.527830+0000","last_unstale":"2024-04-25T00:00:55.657192+0000","last_undegraded":"2024-04-25T00:00:55.657192+0000","last_fullsized":"2024-04-25T00:00:55.657192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:46.846244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653991+0000","last_change":"2024-04-25T00:00:03.143505+0000","last_active":"2024-04-25T00:00:55.653991+0000","last_peered":"2024-04-25T00:00:55.653991+0000","last_clean":"2024-04-25T00:00:55.653991+0000","last_became_active":"2024-04-25T00:00:03.143131+0000","last_became_peered":"2024-04-25T00:00:03.143131+0000","last_unstale":"2024-04-25T00:00:55.653991+0000","last_undegraded":"2024-04-25T00:00:55.653991+0000","last_fullsized":"2024-04-25T00:00:55.653991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:23.743162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:20.668401+0000","last_change":"2024-04-25T00:00:02.525978+0000","last_active":"2024-04-25T00:01:20.668401+0000","last_peered":"2024-04-25T00:01:20.668401+0000","last_clean":"2024-04-25T00:01:20.668401+0000","last_became_active":"2024-04-25T00:00:02.525641+0000","last_became_peered":"2024-04-25T00:00:02.525641+0000","last_unstale":"2024-04-25T00:01:20.668401+0000","last_undegraded":"2024-04-25T00:01:20.668401+0000","last_fullsized":"2024-04-25T00:01:20.668401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:58.508573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656866+0000","last_change":"2024-04-25T00:00:03.143734+0000","last_active":"2024-04-25T00:00:55.656866+0000","last_peered":"2024-04-25T00:00:55.656866+0000","last_clean":"2024-04-25T00:00:55.656866+0000","last_became_active":"2024-04-25T00:00:03.143402+0000","last_became_peered":"2024-04-25T00:00:03.143402+0000","last_unstale":"2024-04-25T00:00:55.656866+0000","last_undegraded":"2024-04-25T00:00:55.656866+0000","last_fullsized":"2024-04-25T00:00:55.656866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:45:36.608803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654913+0000","last_change":"2024-04-25T00:00:02.530567+0000","last_active":"2024-04-25T00:00:55.654913+0000","last_peered":"2024-04-25T00:00:55.654913+0000","last_clean":"2024-04-25T00:00:55.654913+0000","last_became_active":"2024-04-25T00:00:02.530066+0000","last_became_peered":"2024-04-25T00:00:02.530066+0000","last_unstale":"2024-04-25T00:00:55.654913+0000","last_undegraded":"2024-04-25T00:00:55.654913+0000","last_fullsized":"2024-04-25T00:00:55.654913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:19.892052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651530+0000","last_change":"2024-04-25T00:00:03.146905+0000","last_active":"2024-04-25T00:00:55.651530+0000","last_peered":"2024-04-25T00:00:55.651530+0000","last_clean":"2024-04-25T00:00:55.651530+0000","last_became_active":"2024-04-25T00:00:03.146080+0000","last_became_peered":"2024-04-25T00:00:03.146080+0000","last_unstale":"2024-04-25T00:00:55.651530+0000","last_undegraded":"2024-04-25T00:00:55.651530+0000","last_fullsized":"2024-04-25T00:00:55.651530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:42.764082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.658095+0000","last_change":"2024-04-25T00:00:02.527667+0000","last_active":"2024-04-25T00:00:55.658095+0000","last_peered":"2024-04-25T00:00:55.658095+0000","last_clean":"2024-04-25T00:00:55.658095+0000","last_became_active":"2024-04-25T00:00:02.527333+0000","last_became_peered":"2024-04-25T00:00:02.527333+0000","last_unstale":"2024-04-25T00:00:55.658095+0000","last_undegraded":"2024-04-25T00:00:55.658095+0000","last_fullsized":"2024-04-25T00:00:55.658095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:15.982704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656832+0000","last_change":"2024-04-25T00:00:03.145856+0000","last_active":"2024-04-25T00:00:55.656832+0000","last_peered":"2024-04-25T00:00:55.656832+0000","last_clean":"2024-04-25T00:00:55.656832+0000","last_became_active":"2024-04-25T00:00:03.145406+0000","last_became_peered":"2024-04-25T00:00:03.145406+0000","last_unstale":"2024-04-25T00:00:55.656832+0000","last_undegraded":"2024-04-25T00:00:55.656832+0000","last_fullsized":"2024-04-25T00:00:55.656832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:28.318575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":205,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:21.669914+0000","last_change":"2024-04-25T00:00:02.522294+0000","last_active":"2024-04-25T00:01:21.669914+0000","last_peered":"2024-04-25T00:01:21.669914+0000","last_clean":"2024-04-25T00:01:21.669914+0000","last_became_active":"2024-04-25T00:00:02.521776+0000","last_became_peered":"2024-04-25T00:00:02.521776+0000","last_unstale":"2024-04-25T00:01:21.669914+0000","last_undegraded":"2024-04-25T00:01:21.669914+0000","last_fullsized":"2024-04-25T00:01:21.669914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:07.677118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58766,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":158,"num_read_kb":135,"num_write":29,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652252+0000","last_change":"2024-04-25T00:00:03.143877+0000","last_active":"2024-04-25T00:00:55.652252+0000","last_peered":"2024-04-25T00:00:55.652252+0000","last_clean":"2024-04-25T00:00:55.652252+0000","last_became_active":"2024-04-25T00:00:03.143558+0000","last_became_peered":"2024-04-25T00:00:03.143558+0000","last_unstale":"2024-04-25T00:00:55.652252+0000","last_undegraded":"2024-04-25T00:00:55.652252+0000","last_fullsized":"2024-04-25T00:00:55.652252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:37.306175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656397+0000","last_change":"2024-04-25T00:00:02.520760+0000","last_active":"2024-04-25T00:00:55.656397+0000","last_peered":"2024-04-25T00:00:55.656397+0000","last_clean":"2024-04-25T00:00:55.656397+0000","last_became_active":"2024-04-25T00:00:02.520417+0000","last_became_peered":"2024-04-25T00:00:02.520417+0000","last_unstale":"2024-04-25T00:00:55.656397+0000","last_undegraded":"2024-04-25T00:00:55.656397+0000","last_fullsized":"2024-04-25T00:00:55.656397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:53.895653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654534+0000","last_change":"2024-04-25T00:00:03.144157+0000","last_active":"2024-04-25T00:00:55.654534+0000","last_peered":"2024-04-25T00:00:55.654534+0000","last_clean":"2024-04-25T00:00:55.654534+0000","last_became_active":"2024-04-25T00:00:03.143816+0000","last_became_peered":"2024-04-25T00:00:03.143816+0000","last_unstale":"2024-04-25T00:00:55.654534+0000","last_undegraded":"2024-04-25T00:00:55.654534+0000","last_fullsized":"2024-04-25T00:00:55.654534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:45.007842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657177+0000","last_change":"2024-04-25T00:00:02.534393+0000","last_active":"2024-04-25T00:00:55.657177+0000","last_peered":"2024-04-25T00:00:55.657177+0000","last_clean":"2024-04-25T00:00:55.657177+0000","last_became_active":"2024-04-25T00:00:02.534045+0000","last_became_peered":"2024-04-25T00:00:02.534045+0000","last_unstale":"2024-04-25T00:00:55.657177+0000","last_undegraded":"2024-04-25T00:00:55.657177+0000","last_fullsized":"2024-04-25T00:00:55.657177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:35.547065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657375+0000","last_change":"2024-04-25T00:00:55.657375+0000","last_active":"2024-04-25T00:00:55.657375+0000","last_peered":"2024-04-25T00:00:55.657375+0000","last_clean":"2024-04-25T00:00:55.657375+0000","last_became_active":"2024-04-25T00:00:55.656760+0000","last_became_peered":"2024-04-25T00:00:55.656760+0000","last_unstale":"2024-04-25T00:00:55.657375+0000","last_undegraded":"2024-04-25T00:00:55.657375+0000","last_fullsized":"2024-04-25T00:00:55.657375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:34.296182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":101,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:20.685993+0000","last_change":"2024-04-25T00:00:02.520357+0000","last_active":"2024-04-25T00:01:20.685993+0000","last_peered":"2024-04-25T00:01:20.685993+0000","last_clean":"2024-04-25T00:01:20.685993+0000","last_became_active":"2024-04-25T00:00:02.519926+0000","last_became_peered":"2024-04-25T00:00:02.519926+0000","last_unstale":"2024-04-25T00:01:20.685993+0000","last_undegraded":"2024-04-25T00:01:20.685993+0000","last_fullsized":"2024-04-25T00:01:20.685993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:52.438584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653891+0000","last_change":"2024-04-25T00:00:03.147560+0000","last_active":"2024-04-25T00:00:55.653891+0000","last_peered":"2024-04-25T00:00:55.653891+0000","last_clean":"2024-04-25T00:00:55.653891+0000","last_became_active":"2024-04-25T00:00:03.147234+0000","last_became_peered":"2024-04-25T00:00:03.147234+0000","last_unstale":"2024-04-25T00:00:55.653891+0000","last_undegraded":"2024-04-25T00:00:55.653891+0000","last_fullsized":"2024-04-25T00:00:55.653891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:33.737763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651858+0000","last_change":"2024-04-25T00:00:02.528614+0000","last_active":"2024-04-25T00:00:55.651858+0000","last_peered":"2024-04-25T00:00:55.651858+0000","last_clean":"2024-04-25T00:00:55.651858+0000","last_became_active":"2024-04-25T00:00:02.528203+0000","last_became_peered":"2024-04-25T00:00:02.528203+0000","last_unstale":"2024-04-25T00:00:55.651858+0000","last_undegraded":"2024-04-25T00:00:55.651858+0000","last_fullsized":"2024-04-25T00:00:55.651858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:33.116699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656908+0000","last_change":"2024-04-25T00:00:03.145009+0000","last_active":"2024-04-25T00:00:55.656908+0000","last_peered":"2024-04-25T00:00:55.656908+0000","last_clean":"2024-04-25T00:00:55.656908+0000","last_became_active":"2024-04-25T00:00:03.144575+0000","last_became_peered":"2024-04-25T00:00:03.144575+0000","last_unstale":"2024-04-25T00:00:55.656908+0000","last_undegraded":"2024-04-25T00:00:55.656908+0000","last_fullsized":"2024-04-25T00:00:55.656908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.219880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652436+0000","last_change":"2024-04-25T00:00:02.524951+0000","last_active":"2024-04-25T00:00:55.652436+0000","last_peered":"2024-04-25T00:00:55.652436+0000","last_clean":"2024-04-25T00:00:55.652436+0000","last_became_active":"2024-04-25T00:00:02.524624+0000","last_became_peered":"2024-04-25T00:00:02.524624+0000","last_unstale":"2024-04-25T00:00:55.652436+0000","last_undegraded":"2024-04-25T00:00:55.652436+0000","last_fullsized":"2024-04-25T00:00:55.652436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:23.381521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653872+0000","last_change":"2024-04-25T00:00:03.145446+0000","last_active":"2024-04-25T00:00:55.653872+0000","last_peered":"2024-04-25T00:00:55.653872+0000","last_clean":"2024-04-25T00:00:55.653872+0000","last_became_active":"2024-04-25T00:00:03.145125+0000","last_became_peered":"2024-04-25T00:00:03.145125+0000","last_unstale":"2024-04-25T00:00:55.653872+0000","last_undegraded":"2024-04-25T00:00:55.653872+0000","last_fullsized":"2024-04-25T00:00:55.653872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:40.187270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654370+0000","last_change":"2024-04-25T00:00:02.527499+0000","last_active":"2024-04-25T00:00:55.654370+0000","last_peered":"2024-04-25T00:00:55.654370+0000","last_clean":"2024-04-25T00:00:55.654370+0000","last_became_active":"2024-04-25T00:00:02.527151+0000","last_became_peered":"2024-04-25T00:00:02.527151+0000","last_unstale":"2024-04-25T00:00:55.654370+0000","last_undegraded":"2024-04-25T00:00:55.654370+0000","last_fullsized":"2024-04-25T00:00:55.654370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:09.013930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017381+0000","last_change":"2024-04-25T00:00:03.148231+0000","last_active":"2024-04-25T00:00:56.017381+0000","last_peered":"2024-04-25T00:00:56.017381+0000","last_clean":"2024-04-25T00:00:56.017381+0000","last_became_active":"2024-04-25T00:00:03.147891+0000","last_became_peered":"2024-04-25T00:00:03.147891+0000","last_unstale":"2024-04-25T00:00:56.017381+0000","last_undegraded":"2024-04-25T00:00:56.017381+0000","last_fullsized":"2024-04-25T00:00:56.017381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:42.784699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654156+0000","last_change":"2024-04-25T00:00:02.528234+0000","last_active":"2024-04-25T00:00:55.654156+0000","last_peered":"2024-04-25T00:00:55.654156+0000","last_clean":"2024-04-25T00:00:55.654156+0000","last_became_active":"2024-04-25T00:00:02.527875+0000","last_became_peered":"2024-04-25T00:00:02.527875+0000","last_unstale":"2024-04-25T00:00:55.654156+0000","last_undegraded":"2024-04-25T00:00:55.654156+0000","last_fullsized":"2024-04-25T00:00:55.654156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:06.634497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:56.017250+0000","last_change":"2024-04-25T00:00:03.144246+0000","last_active":"2024-04-25T00:00:56.017250+0000","last_peered":"2024-04-25T00:00:56.017250+0000","last_clean":"2024-04-25T00:00:56.017250+0000","last_became_active":"2024-04-25T00:00:03.143925+0000","last_became_peered":"2024-04-25T00:00:03.143925+0000","last_unstale":"2024-04-25T00:00:56.017250+0000","last_undegraded":"2024-04-25T00:00:56.017250+0000","last_fullsized":"2024-04-25T00:00:56.017250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:51.741275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657464+0000","last_change":"2024-04-25T00:00:03.141261+0000","last_active":"2024-04-25T00:00:55.657464+0000","last_peered":"2024-04-25T00:00:55.657464+0000","last_clean":"2024-04-25T00:00:55.657464+0000","last_became_active":"2024-04-25T00:00:03.137943+0000","last_became_peered":"2024-04-25T00:00:03.137943+0000","last_unstale":"2024-04-25T00:00:55.657464+0000","last_undegraded":"2024-04-25T00:00:55.657464+0000","last_fullsized":"2024-04-25T00:00:55.657464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:41.655749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:01:20.685219+0000","last_change":"2024-04-25T00:00:02.522048+0000","last_active":"2024-04-25T00:01:20.685219+0000","last_peered":"2024-04-25T00:01:20.685219+0000","last_clean":"2024-04-25T00:01:20.685219+0000","last_became_active":"2024-04-25T00:00:02.521755+0000","last_became_peered":"2024-04-25T00:00:02.521755+0000","last_unstale":"2024-04-25T00:01:20.685219+0000","last_undegraded":"2024-04-25T00:01:20.685219+0000","last_fullsized":"2024-04-25T00:01:20.685219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:58.602357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656389+0000","last_change":"2024-04-25T00:00:03.151428+0000","last_active":"2024-04-25T00:00:55.656389+0000","last_peered":"2024-04-25T00:00:55.656389+0000","last_clean":"2024-04-25T00:00:55.656389+0000","last_became_active":"2024-04-25T00:00:03.151005+0000","last_became_peered":"2024-04-25T00:00:03.151005+0000","last_unstale":"2024-04-25T00:00:55.656389+0000","last_undegraded":"2024-04-25T00:00:55.656389+0000","last_fullsized":"2024-04-25T00:00:55.656389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:28.044803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653428+0000","last_change":"2024-04-25T00:00:02.515714+0000","last_active":"2024-04-25T00:00:55.653428+0000","last_peered":"2024-04-25T00:00:55.653428+0000","last_clean":"2024-04-25T00:00:55.653428+0000","last_became_active":"2024-04-25T00:00:02.515373+0000","last_became_peered":"2024-04-25T00:00:02.515373+0000","last_unstale":"2024-04-25T00:00:55.653428+0000","last_undegraded":"2024-04-25T00:00:55.653428+0000","last_fullsized":"2024-04-25T00:00:55.653428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:11:34.344261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654014+0000","last_change":"2024-04-24T23:58:50.595622+0000","last_active":"2024-04-25T00:00:55.654014+0000","last_peered":"2024-04-25T00:00:55.654014+0000","last_clean":"2024-04-25T00:00:55.654014+0000","last_became_active":"2024-04-24T23:58:50.594971+0000","last_became_peered":"2024-04-24T23:58:50.594971+0000","last_unstale":"2024-04-25T00:00:55.654014+0000","last_undegraded":"2024-04-25T00:00:55.654014+0000","last_fullsized":"2024-04-25T00:00:55.654014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:55:22.756734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:55:22.756734+0000","last_clean_scrub_stamp":"2024-04-24T23:55:22.756734+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:17:20.241007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653794+0000","last_change":"2024-04-25T00:00:03.135594+0000","last_active":"2024-04-25T00:00:55.653794+0000","last_peered":"2024-04-25T00:00:55.653794+0000","last_clean":"2024-04-25T00:00:55.653794+0000","last_became_active":"2024-04-25T00:00:03.135279+0000","last_became_peered":"2024-04-25T00:00:03.135279+0000","last_unstale":"2024-04-25T00:00:55.653794+0000","last_undegraded":"2024-04-25T00:00:55.653794+0000","last_fullsized":"2024-04-25T00:00:55.653794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:37.690119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.663160+0000","last_change":"2024-04-25T00:00:55.663160+0000","last_active":"2024-04-25T00:00:55.663160+0000","last_peered":"2024-04-25T00:00:55.663160+0000","last_clean":"2024-04-25T00:00:55.663160+0000","last_became_active":"2024-04-25T00:00:55.659082+0000","last_became_peered":"2024-04-25T00:00:55.659082+0000","last_unstale":"2024-04-25T00:00:55.663160+0000","last_undegraded":"2024-04-25T00:00:55.663160+0000","last_fullsized":"2024-04-25T00:00:55.663160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:20.586443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652407+0000","last_change":"2024-04-25T00:00:02.527545+0000","last_active":"2024-04-25T00:00:55.652407+0000","last_peered":"2024-04-25T00:00:55.652407+0000","last_clean":"2024-04-25T00:00:55.652407+0000","last_became_active":"2024-04-25T00:00:02.527250+0000","last_became_peered":"2024-04-25T00:00:02.527250+0000","last_unstale":"2024-04-25T00:00:55.652407+0000","last_undegraded":"2024-04-25T00:00:55.652407+0000","last_fullsized":"2024-04-25T00:00:55.652407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:04:57.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654599+0000","last_change":"2024-04-25T00:00:03.139578+0000","last_active":"2024-04-25T00:00:55.654599+0000","last_peered":"2024-04-25T00:00:55.654599+0000","last_clean":"2024-04-25T00:00:55.654599+0000","last_became_active":"2024-04-25T00:00:03.139128+0000","last_became_peered":"2024-04-25T00:00:03.139128+0000","last_unstale":"2024-04-25T00:00:55.654599+0000","last_undegraded":"2024-04-25T00:00:55.654599+0000","last_fullsized":"2024-04-25T00:00:55.654599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:33.509293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654125+0000","last_change":"2024-04-25T00:00:02.520616+0000","last_active":"2024-04-25T00:00:55.654125+0000","last_peered":"2024-04-25T00:00:55.654125+0000","last_clean":"2024-04-25T00:00:55.654125+0000","last_became_active":"2024-04-25T00:00:02.519057+0000","last_became_peered":"2024-04-25T00:00:02.519057+0000","last_unstale":"2024-04-25T00:00:55.654125+0000","last_undegraded":"2024-04-25T00:00:55.654125+0000","last_fullsized":"2024-04-25T00:00:55.654125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:25.847055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656655+0000","last_change":"2024-04-25T00:00:03.150010+0000","last_active":"2024-04-25T00:00:55.656655+0000","last_peered":"2024-04-25T00:00:55.656655+0000","last_clean":"2024-04-25T00:00:55.656655+0000","last_became_active":"2024-04-25T00:00:03.149677+0000","last_became_peered":"2024-04-25T00:00:03.149677+0000","last_unstale":"2024-04-25T00:00:55.656655+0000","last_undegraded":"2024-04-25T00:00:55.656655+0000","last_fullsized":"2024-04-25T00:00:55.656655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:18.050720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653112+0000","last_change":"2024-04-25T00:00:02.533171+0000","last_active":"2024-04-25T00:00:55.653112+0000","last_peered":"2024-04-25T00:00:55.653112+0000","last_clean":"2024-04-25T00:00:55.653112+0000","last_became_active":"2024-04-25T00:00:02.532887+0000","last_became_peered":"2024-04-25T00:00:02.532887+0000","last_unstale":"2024-04-25T00:00:55.653112+0000","last_undegraded":"2024-04-25T00:00:55.653112+0000","last_fullsized":"2024-04-25T00:00:55.653112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:41:24.729809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651467+0000","last_change":"2024-04-25T00:00:03.147457+0000","last_active":"2024-04-25T00:00:55.651467+0000","last_peered":"2024-04-25T00:00:55.651467+0000","last_clean":"2024-04-25T00:00:55.651467+0000","last_became_active":"2024-04-25T00:00:03.147163+0000","last_became_peered":"2024-04-25T00:00:03.147163+0000","last_unstale":"2024-04-25T00:00:55.651467+0000","last_undegraded":"2024-04-25T00:00:55.651467+0000","last_fullsized":"2024-04-25T00:00:55.651467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:32.422006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656376+0000","last_change":"2024-04-25T00:00:02.527573+0000","last_active":"2024-04-25T00:00:55.656376+0000","last_peered":"2024-04-25T00:00:55.656376+0000","last_clean":"2024-04-25T00:00:55.656376+0000","last_became_active":"2024-04-25T00:00:02.527267+0000","last_became_peered":"2024-04-25T00:00:02.527267+0000","last_unstale":"2024-04-25T00:00:55.656376+0000","last_undegraded":"2024-04-25T00:00:55.656376+0000","last_fullsized":"2024-04-25T00:00:55.656376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:31.581529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651843+0000","last_change":"2024-04-25T00:00:03.146602+0000","last_active":"2024-04-25T00:00:55.651843+0000","last_peered":"2024-04-25T00:00:55.651843+0000","last_clean":"2024-04-25T00:00:55.651843+0000","last_became_active":"2024-04-25T00:00:03.145480+0000","last_became_peered":"2024-04-25T00:00:03.145480+0000","last_unstale":"2024-04-25T00:00:55.651843+0000","last_undegraded":"2024-04-25T00:00:55.651843+0000","last_fullsized":"2024-04-25T00:00:55.651843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:02.803903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656153+0000","last_change":"2024-04-25T00:00:02.526567+0000","last_active":"2024-04-25T00:00:55.656153+0000","last_peered":"2024-04-25T00:00:55.656153+0000","last_clean":"2024-04-25T00:00:55.656153+0000","last_became_active":"2024-04-25T00:00:02.526234+0000","last_became_peered":"2024-04-25T00:00:02.526234+0000","last_unstale":"2024-04-25T00:00:55.656153+0000","last_undegraded":"2024-04-25T00:00:55.656153+0000","last_fullsized":"2024-04-25T00:00:55.656153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:14.964206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653520+0000","last_change":"2024-04-25T00:00:03.140959+0000","last_active":"2024-04-25T00:00:55.653520+0000","last_peered":"2024-04-25T00:00:55.653520+0000","last_clean":"2024-04-25T00:00:55.653520+0000","last_became_active":"2024-04-25T00:00:03.140533+0000","last_became_peered":"2024-04-25T00:00:03.140533+0000","last_unstale":"2024-04-25T00:00:55.653520+0000","last_undegraded":"2024-04-25T00:00:55.653520+0000","last_fullsized":"2024-04-25T00:00:55.653520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:46.166051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.664257+0000","last_change":"2024-04-25T00:00:55.663762+0000","last_active":"2024-04-25T00:00:55.664257+0000","last_peered":"2024-04-25T00:00:55.664257+0000","last_clean":"2024-04-25T00:00:55.664257+0000","last_became_active":"2024-04-25T00:00:55.657906+0000","last_became_peered":"2024-04-25T00:00:55.657906+0000","last_unstale":"2024-04-25T00:00:55.664257+0000","last_undegraded":"2024-04-25T00:00:55.664257+0000","last_fullsized":"2024-04-25T00:00:55.664257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:42.935465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656464+0000","last_change":"2024-04-25T00:00:03.149740+0000","last_active":"2024-04-25T00:00:55.656464+0000","last_peered":"2024-04-25T00:00:55.656464+0000","last_clean":"2024-04-25T00:00:55.656464+0000","last_became_active":"2024-04-25T00:00:03.149290+0000","last_became_peered":"2024-04-25T00:00:03.149290+0000","last_unstale":"2024-04-25T00:00:55.656464+0000","last_undegraded":"2024-04-25T00:00:55.656464+0000","last_fullsized":"2024-04-25T00:00:55.656464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:15.716974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.652063+0000","last_change":"2024-04-25T00:00:02.526823+0000","last_active":"2024-04-25T00:00:55.652063+0000","last_peered":"2024-04-25T00:00:55.652063+0000","last_clean":"2024-04-25T00:00:55.652063+0000","last_became_active":"2024-04-25T00:00:02.526529+0000","last_became_peered":"2024-04-25T00:00:02.526529+0000","last_unstale":"2024-04-25T00:00:55.652063+0000","last_undegraded":"2024-04-25T00:00:55.652063+0000","last_fullsized":"2024-04-25T00:00:55.652063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:25:13.907985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654711+0000","last_change":"2024-04-25T00:00:03.141973+0000","last_active":"2024-04-25T00:00:55.654711+0000","last_peered":"2024-04-25T00:00:55.654711+0000","last_clean":"2024-04-25T00:00:55.654711+0000","last_became_active":"2024-04-25T00:00:03.141560+0000","last_became_peered":"2024-04-25T00:00:03.141560+0000","last_unstale":"2024-04-25T00:00:55.654711+0000","last_undegraded":"2024-04-25T00:00:55.654711+0000","last_fullsized":"2024-04-25T00:00:55.654711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:03.002939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657504+0000","last_change":"2024-04-25T00:00:02.533013+0000","last_active":"2024-04-25T00:00:55.657504+0000","last_peered":"2024-04-25T00:00:55.657504+0000","last_clean":"2024-04-25T00:00:55.657504+0000","last_became_active":"2024-04-25T00:00:02.532667+0000","last_became_peered":"2024-04-25T00:00:02.532667+0000","last_unstale":"2024-04-25T00:00:55.657504+0000","last_undegraded":"2024-04-25T00:00:55.657504+0000","last_fullsized":"2024-04-25T00:00:55.657504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:07.469136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654548+0000","last_change":"2024-04-25T00:00:03.140120+0000","last_active":"2024-04-25T00:00:55.654548+0000","last_peered":"2024-04-25T00:00:55.654548+0000","last_clean":"2024-04-25T00:00:55.654548+0000","last_became_active":"2024-04-25T00:00:03.139778+0000","last_became_peered":"2024-04-25T00:00:03.139778+0000","last_unstale":"2024-04-25T00:00:55.654548+0000","last_undegraded":"2024-04-25T00:00:55.654548+0000","last_fullsized":"2024-04-25T00:00:55.654548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:02.678303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657422+0000","last_change":"2024-04-25T00:00:02.534359+0000","last_active":"2024-04-25T00:00:55.657422+0000","last_peered":"2024-04-25T00:00:55.657422+0000","last_clean":"2024-04-25T00:00:55.657422+0000","last_became_active":"2024-04-25T00:00:02.534057+0000","last_became_peered":"2024-04-25T00:00:02.534057+0000","last_unstale":"2024-04-25T00:00:55.657422+0000","last_undegraded":"2024-04-25T00:00:55.657422+0000","last_fullsized":"2024-04-25T00:00:55.657422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:38.716263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654205+0000","last_change":"2024-04-25T00:00:03.146608+0000","last_active":"2024-04-25T00:00:55.654205+0000","last_peered":"2024-04-25T00:00:55.654205+0000","last_clean":"2024-04-25T00:00:55.654205+0000","last_became_active":"2024-04-25T00:00:03.146256+0000","last_became_peered":"2024-04-25T00:00:03.146256+0000","last_unstale":"2024-04-25T00:00:55.654205+0000","last_undegraded":"2024-04-25T00:00:55.654205+0000","last_fullsized":"2024-04-25T00:00:55.654205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:04.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.661162+0000","last_change":"2024-04-25T00:00:55.661162+0000","last_active":"2024-04-25T00:00:55.661162+0000","last_peered":"2024-04-25T00:00:55.661162+0000","last_clean":"2024-04-25T00:00:55.661162+0000","last_became_active":"2024-04-25T00:00:55.657844+0000","last_became_peered":"2024-04-25T00:00:55.657844+0000","last_unstale":"2024-04-25T00:00:55.661162+0000","last_undegraded":"2024-04-25T00:00:55.661162+0000","last_fullsized":"2024-04-25T00:00:55.661162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:01.501429+0000","last_clean_scrub_stamp":"2024-04-25T00:00:01.501429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:18.458244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654438+0000","last_change":"2024-04-25T00:00:03.141296+0000","last_active":"2024-04-25T00:00:55.654438+0000","last_peered":"2024-04-25T00:00:55.654438+0000","last_clean":"2024-04-25T00:00:55.654438+0000","last_became_active":"2024-04-25T00:00:03.139916+0000","last_became_peered":"2024-04-25T00:00:03.139916+0000","last_unstale":"2024-04-25T00:00:55.654438+0000","last_undegraded":"2024-04-25T00:00:55.654438+0000","last_fullsized":"2024-04-25T00:00:55.654438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:24.534210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653635+0000","last_change":"2024-04-25T00:00:03.149427+0000","last_active":"2024-04-25T00:00:55.653635+0000","last_peered":"2024-04-25T00:00:55.653635+0000","last_clean":"2024-04-25T00:00:55.653635+0000","last_became_active":"2024-04-25T00:00:03.148445+0000","last_became_peered":"2024-04-25T00:00:03.148445+0000","last_unstale":"2024-04-25T00:00:55.653635+0000","last_undegraded":"2024-04-25T00:00:55.653635+0000","last_fullsized":"2024-04-25T00:00:55.653635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:35.212153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657285+0000","last_change":"2024-04-25T00:00:03.146994+0000","last_active":"2024-04-25T00:00:55.657285+0000","last_peered":"2024-04-25T00:00:55.657285+0000","last_clean":"2024-04-25T00:00:55.657285+0000","last_became_active":"2024-04-25T00:00:03.146677+0000","last_became_peered":"2024-04-25T00:00:03.146677+0000","last_unstale":"2024-04-25T00:00:55.657285+0000","last_undegraded":"2024-04-25T00:00:55.657285+0000","last_fullsized":"2024-04-25T00:00:55.657285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:33.883583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656381+0000","last_change":"2024-04-25T00:00:03.149428+0000","last_active":"2024-04-25T00:00:55.656381+0000","last_peered":"2024-04-25T00:00:55.656381+0000","last_clean":"2024-04-25T00:00:55.656381+0000","last_became_active":"2024-04-25T00:00:03.147978+0000","last_became_peered":"2024-04-25T00:00:03.147978+0000","last_unstale":"2024-04-25T00:00:55.656381+0000","last_undegraded":"2024-04-25T00:00:55.656381+0000","last_fullsized":"2024-04-25T00:00:55.656381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:47:34.684463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.651336+0000","last_change":"2024-04-25T00:00:03.147528+0000","last_active":"2024-04-25T00:00:55.651336+0000","last_peered":"2024-04-25T00:00:55.651336+0000","last_clean":"2024-04-25T00:00:55.651336+0000","last_became_active":"2024-04-25T00:00:03.147182+0000","last_became_peered":"2024-04-25T00:00:03.147182+0000","last_unstale":"2024-04-25T00:00:55.651336+0000","last_undegraded":"2024-04-25T00:00:55.651336+0000","last_fullsized":"2024-04-25T00:00:55.651336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:27.592335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654696+0000","last_change":"2024-04-25T00:00:03.143274+0000","last_active":"2024-04-25T00:00:55.654696+0000","last_peered":"2024-04-25T00:00:55.654696+0000","last_clean":"2024-04-25T00:00:55.654696+0000","last_became_active":"2024-04-25T00:00:03.142384+0000","last_became_peered":"2024-04-25T00:00:03.142384+0000","last_unstale":"2024-04-25T00:00:55.654696+0000","last_undegraded":"2024-04-25T00:00:55.654696+0000","last_fullsized":"2024-04-25T00:00:55.654696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:41.118123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.653696+0000","last_change":"2024-04-25T00:00:03.147769+0000","last_active":"2024-04-25T00:00:55.653696+0000","last_peered":"2024-04-25T00:00:55.653696+0000","last_clean":"2024-04-25T00:00:55.653696+0000","last_became_active":"2024-04-25T00:00:03.147009+0000","last_became_peered":"2024-04-25T00:00:03.147009+0000","last_unstale":"2024-04-25T00:00:55.653696+0000","last_undegraded":"2024-04-25T00:00:55.653696+0000","last_fullsized":"2024-04-25T00:00:55.653696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:18.917532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.658543+0000","last_change":"2024-04-25T00:00:03.141519+0000","last_active":"2024-04-25T00:00:55.658543+0000","last_peered":"2024-04-25T00:00:55.658543+0000","last_clean":"2024-04-25T00:00:55.658543+0000","last_became_active":"2024-04-25T00:00:03.139391+0000","last_became_peered":"2024-04-25T00:00:03.139391+0000","last_unstale":"2024-04-25T00:00:55.658543+0000","last_undegraded":"2024-04-25T00:00:55.658543+0000","last_fullsized":"2024-04-25T00:00:55.658543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:13.043899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656618+0000","last_change":"2024-04-25T00:00:03.151311+0000","last_active":"2024-04-25T00:00:55.656618+0000","last_peered":"2024-04-25T00:00:55.656618+0000","last_clean":"2024-04-25T00:00:55.656618+0000","last_became_active":"2024-04-25T00:00:03.150992+0000","last_became_peered":"2024-04-25T00:00:03.150992+0000","last_unstale":"2024-04-25T00:00:55.656618+0000","last_undegraded":"2024-04-25T00:00:55.656618+0000","last_fullsized":"2024-04-25T00:00:55.656618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:47.572641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.657364+0000","last_change":"2024-04-25T00:00:03.145623+0000","last_active":"2024-04-25T00:00:55.657364+0000","last_peered":"2024-04-25T00:00:55.657364+0000","last_clean":"2024-04-25T00:00:55.657364+0000","last_became_active":"2024-04-25T00:00:03.145260+0000","last_became_peered":"2024-04-25T00:00:03.145260+0000","last_unstale":"2024-04-25T00:00:55.657364+0000","last_undegraded":"2024-04-25T00:00:55.657364+0000","last_fullsized":"2024-04-25T00:00:55.657364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:16.985732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656105+0000","last_change":"2024-04-25T00:00:03.144957+0000","last_active":"2024-04-25T00:00:55.656105+0000","last_peered":"2024-04-25T00:00:55.656105+0000","last_clean":"2024-04-25T00:00:55.656105+0000","last_became_active":"2024-04-25T00:00:03.144631+0000","last_became_peered":"2024-04-25T00:00:03.144631+0000","last_unstale":"2024-04-25T00:00:55.656105+0000","last_undegraded":"2024-04-25T00:00:55.656105+0000","last_fullsized":"2024-04-25T00:00:55.656105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.636553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.656487+0000","last_change":"2024-04-25T00:00:03.146574+0000","last_active":"2024-04-25T00:00:55.656487+0000","last_peered":"2024-04-25T00:00:55.656487+0000","last_clean":"2024-04-25T00:00:55.656487+0000","last_became_active":"2024-04-25T00:00:03.145095+0000","last_became_peered":"2024-04-25T00:00:03.145095+0000","last_unstale":"2024-04-25T00:00:55.656487+0000","last_undegraded":"2024-04-25T00:00:55.656487+0000","last_fullsized":"2024-04-25T00:00:55.656487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:38.060170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654352+0000","last_change":"2024-04-25T00:00:03.143958+0000","last_active":"2024-04-25T00:00:55.654352+0000","last_peered":"2024-04-25T00:00:55.654352+0000","last_clean":"2024-04-25T00:00:55.654352+0000","last_became_active":"2024-04-25T00:00:03.143621+0000","last_became_peered":"2024-04-25T00:00:03.143621+0000","last_unstale":"2024-04-25T00:00:55.654352+0000","last_undegraded":"2024-04-25T00:00:55.654352+0000","last_fullsized":"2024-04-25T00:00:55.654352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:21.833252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T00:00:55.654417+0000","last_change":"2024-04-25T00:00:03.143703+0000","last_active":"2024-04-25T00:00:55.654417+0000","last_peered":"2024-04-25T00:00:55.654417+0000","last_clean":"2024-04-25T00:00:55.654417+0000","last_became_active":"2024-04-25T00:00:03.143400+0000","last_became_peered":"2024-04-25T00:00:03.143400+0000","last_unstale":"2024-04-25T00:00:55.654417+0000","last_undegraded":"2024-04-25T00:00:55.654417+0000","last_fullsized":"2024-04-25T00:00:55.654417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:00:02.506088+0000","last_clean_scrub_stamp":"2024-04-25T00:00:02.506088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:53.984769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284249,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63158,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":478,"num_read_kb":305,"num_write":157,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":189474,"data_compressed":5559,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":456102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645347,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":1106740,"data_compressed":37998,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1048579,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":455727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":514286,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":456998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":516027,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":456038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":456555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":1047872,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34508,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58766,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60507,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60735,"data_compressed":1853,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:01:26.518 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T00:01:26.518 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:01:26.518 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:01:26.518 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T00:01:26.760 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:27.955 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:27 smithi110 ceph-mon[29187]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:01:27.955 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:27 smithi110 ceph-mon[29187]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:01:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:27 smithi188 ceph-mon[33285]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:01:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:27 smithi188 ceph-mon[33285]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:01:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:27 smithi173 ceph-mon[32402]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:01:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:27 smithi173 ceph-mon[32402]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:01:28.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 -- 172.21.15.110:0/127929828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e0101af0 msgr2=0x7ff8e010dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/127929828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e0101af0 0x7ff8e010dfe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800b0a0 tx=0x7ff8d800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 -- 172.21.15.110:0/127929828 shutdown_connections 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/127929828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e0101af0 0x7ff8e010dfe0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/127929828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e01071a0 0x7ff8e0107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/127929828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e0101130 0x7ff8e01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 -- 172.21.15.110:0/127929828 >> 172.21.15.110:0/127929828 conn(0x7ff8e00780e0 msgr2=0x7ff8e00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 -- 172.21.15.110:0/127929828 shutdown_connections 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.558+0000 7ff8e7b63700 1 -- 172.21.15.110:0/127929828 wait complete. 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e7b63700 1 Processor -- start 2024-04-25T00:01:28.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e7b63700 1 -- start start 2024-04-25T00:01:28.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e7b63700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e0101130 0x7ff8e0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e7b63700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0101af0 0x7ff8e0199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e7b63700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e01071a0 0x7ff8e01a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e7b63700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0111340 con 0x7ff8e01071a0 2024-04-25T00:01:28.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e7b63700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01111c0 con 0x7ff8e0101af0 2024-04-25T00:01:28.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e7b63700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01114c0 con 0x7ff8e0101130 2024-04-25T00:01:28.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e6100700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e01071a0 0x7ff8e01a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e6100700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e01071a0 0x7ff8e01a47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60736/0 (socket says 172.21.15.110:60736) 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e6100700 1 -- 172.21.15.110:0/2418643355 learned_addr learned my addr 172.21.15.110:0/2418643355 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e58ff700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e0101130 0x7ff8e0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e50fe700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0101af0 0x7ff8e0199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e6100700 1 -- 172.21.15.110:0/2418643355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e0101130 msgr2=0x7ff8e0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e6100700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e0101130 0x7ff8e0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e6100700 1 -- 172.21.15.110:0/2418643355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0101af0 msgr2=0x7ff8e0199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e6100700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0101af0 0x7ff8e0199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.559+0000 7ff8e6100700 1 -- 172.21.15.110:0/2418643355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e01a4d00 con 0x7ff8e01071a0 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.560+0000 7ff8e6100700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e01071a0 0x7ff8e01a47c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800b830 tx=0x7ff8d8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.560+0000 7ff8d6ffd700 1 -- 172.21.15.110:0/2418643355 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8021070 con 0x7ff8e01071a0 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.560+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e01a4ed0 con 0x7ff8e01071a0 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.560+0000 7ff8d6ffd700 1 -- 172.21.15.110:0/2418643355 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8d8007e30 con 0x7ff8e01071a0 2024-04-25T00:01:28.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.560+0000 7ff8d6ffd700 1 -- 172.21.15.110:0/2418643355 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8004700 con 0x7ff8e01071a0 2024-04-25T00:01:28.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.560+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01a5380 con 0x7ff8e01071a0 2024-04-25T00:01:28.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.561+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e0068f20 con 0x7ff8e01071a0 2024-04-25T00:01:28.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.562+0000 7ff8d6ffd700 1 -- 172.21.15.110:0/2418643355 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff8d8003de0 con 0x7ff8e01071a0 2024-04-25T00:01:28.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.563+0000 7ff8d6ffd700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff8cc0a2c20 0x7ff8cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:28.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.563+0000 7ff8e58ff700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff8cc0a2c20 0x7ff8cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:28.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.563+0000 7ff8d6ffd700 1 -- 172.21.15.110:0/2418643355 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff8d800f020 con 0x7ff8e01071a0 2024-04-25T00:01:28.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.563+0000 7ff8e58ff700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff8cc0a2c20 0x7ff8cc0a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005950 tx=0x7ff8dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:28.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.567+0000 7ff8d6ffd700 1 -- 172.21.15.110:0/2418643355 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8d802a020 con 0x7ff8e01071a0 2024-04-25T00:01:28.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.812+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff8e002dcf0 con 0x7ff8e01071a0 2024-04-25T00:01:28.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.812+0000 7ff8d6ffd700 1 -- 172.21.15.110:0/2418643355 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff8d809d8d0 con 0x7ff8e01071a0 2024-04-25T00:01:28.814 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:28.814 INFO:teuthology.orchestra.run.smithi110.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.814+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff8cc0a2c20 msgr2=0x7ff8cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.814+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff8cc0a2c20 0x7ff8cc0a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005950 tx=0x7ff8dc00b410 comp rx=0 tx=0).stop 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.814+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e01071a0 msgr2=0x7ff8e01a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.814+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e01071a0 0x7ff8e01a47c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800b830 tx=0x7ff8d8004ce0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.815+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 shutdown_connections 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.815+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff8cc0a2c20 0x7ff8cc0a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.815+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8e01071a0 0x7ff8e01a47c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.815+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8e0101130 0x7ff8e0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.815+0000 7ff8e7b63700 1 --2- 172.21.15.110:0/2418643355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0101af0 0x7ff8e0199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.815+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 >> 172.21.15.110:0/2418643355 conn(0x7ff8e00780e0 msgr2=0x7ff8e00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:28.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.815+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 shutdown_connections 2024-04-25T00:01:28.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:28.815+0000 7ff8e7b63700 1 -- 172.21.15.110:0/2418643355 wait complete. 2024-04-25T00:01:29.552 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:01:29.553 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:29.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 -- 172.21.15.110:0/2395580541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878102240 msgr2=0x7f0878108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 --2- 172.21.15.110:0/2395580541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878102240 0x7f0878108ad0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0868009900 tx=0x7f086802a410 comp rx=0 tx=0).stop 2024-04-25T00:01:29.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 -- 172.21.15.110:0/2395580541 shutdown_connections 2024-04-25T00:01:29.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 --2- 172.21.15.110:0/2395580541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878102240 0x7f0878108ad0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 --2- 172.21.15.110:0/2395580541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0878101880 0x7f0878101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 --2- 172.21.15.110:0/2395580541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0878100690 0x7f0878100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 -- 172.21.15.110:0/2395580541 >> 172.21.15.110:0/2395580541 conn(0x7f08780fbc20 msgr2=0x7f08780fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 -- 172.21.15.110:0/2395580541 shutdown_connections 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.657+0000 7f087cebd640 1 -- 172.21.15.110:0/2395580541 wait complete. 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087cebd640 1 Processor -- start 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087cebd640 1 -- start start 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087cebd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0878100690 0x7f0878104620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087cebd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878101880 0x7f0878102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087cebd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0878102240 0x7f08781031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087cebd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f087810c1c0 con 0x7f0878101880 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087cebd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f087810c040 con 0x7f0878102240 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087cebd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f087810c340 con 0x7f0878100690 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0875d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878101880 0x7f0878102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0875d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878101880 0x7f0878102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60760/0 (socket says 172.21.15.110:60760) 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0875d9b640 1 -- 172.21.15.110:0/2083926267 learned_addr learned my addr 172.21.15.110:0/2083926267 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f087659c640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0878100690 0x7f0878104620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0876d9d640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0878102240 0x7f08781031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0875d9b640 1 -- 172.21.15.110:0/2083926267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0878100690 msgr2=0x7f0878104620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0875d9b640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0878100690 0x7f0878104620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0875d9b640 1 -- 172.21.15.110:0/2083926267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0878102240 msgr2=0x7f08781031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0875d9b640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0878102240 0x7f08781031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.658+0000 7f0875d9b640 1 -- 172.21.15.110:0/2083926267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0878103a30 con 0x7f0878101880 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.659+0000 7f0875d9b640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878101880 0x7f0878102c60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f086c00ea80 tx=0x7f086c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.659+0000 7f08677fe640 1 -- 172.21.15.110:0/2083926267 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c00ce40 con 0x7f0878101880 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.659+0000 7f08677fe640 1 -- 172.21.15.110:0/2083926267 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f086c005020 con 0x7f0878101880 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.659+0000 7f08677fe640 1 -- 172.21.15.110:0/2083926267 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0116d0 con 0x7f0878101880 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.659+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08781a1a70 con 0x7f0878101880 2024-04-25T00:01:29.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.659+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08781a1f70 con 0x7f0878101880 2024-04-25T00:01:29.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.660+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08781a2200 con 0x7f0878101880 2024-04-25T00:01:29.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.661+0000 7f08677fe640 1 -- 172.21.15.110:0/2083926267 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f086c0051d0 con 0x7f0878101880 2024-04-25T00:01:29.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.661+0000 7f08677fe640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08600a2c10 0x7f08600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.662+0000 7f087659c640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08600a2c10 0x7f08600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.662+0000 7f08677fe640 1 -- 172.21.15.110:0/2083926267 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f086c0cf220 con 0x7f0878101880 2024-04-25T00:01:29.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.662+0000 7f087659c640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08600a2c10 0x7f08600a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f08781016e0 tx=0x7f0870005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.667+0000 7f08677fe640 1 -- 172.21.15.110:0/2083926267 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f086c016030 con 0x7f0878101880 2024-04-25T00:01:29.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:29 smithi110 ceph-mon[29187]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:01:29.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:29 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2418643355' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:01:29.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.845+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08781a29b0 con 0x7f0878101880 2024-04-25T00:01:29.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.846+0000 7f08677fe640 1 -- 172.21.15.110:0/2083926267 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7f086c09bb60 con 0x7f0878101880 2024-04-25T00:01:29.847 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:29.848 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08600a2c10 msgr2=0x7f08600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08600a2c10 0x7f08600a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f08781016e0 tx=0x7f0870005f90 comp rx=0 tx=0).stop 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878101880 msgr2=0x7f0878102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878101880 0x7f0878102c60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f086c00ea80 tx=0x7f086c00ef40 comp rx=0 tx=0).stop 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 shutdown_connections 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f08600a2c10 0x7f08600a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0878102240 0x7f08781031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0878101880 0x7f0878102c60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 --2- 172.21.15.110:0/2083926267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0878100690 0x7f0878104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 >> 172.21.15.110:0/2083926267 conn(0x7f08780fbc20 msgr2=0x7f08780fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 shutdown_connections 2024-04-25T00:01:29.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.849+0000 7f087cebd640 1 -- 172.21.15.110:0/2083926267 wait complete. 2024-04-25T00:01:29.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:01.224042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:02.090769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:29.862 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 -- 172.21.15.110:0/2864517778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 msgr2=0x7efd30101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 --2- 172.21.15.110:0/2864517778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 0x7efd30101cf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efd20009900 tx=0x7efd2002a410 comp rx=0 tx=0).stop 2024-04-25T00:01:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 -- 172.21.15.110:0/2864517778 shutdown_connections 2024-04-25T00:01:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 --2- 172.21.15.110:0/2864517778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd30102230 0x7efd30108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 --2- 172.21.15.110:0/2864517778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 0x7efd30101cf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 --2- 172.21.15.110:0/2864517778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd30100680 0x7efd30100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 -- 172.21.15.110:0/2864517778 >> 172.21.15.110:0/2864517778 conn(0x7efd300fbbf0 msgr2=0x7efd300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 -- 172.21.15.110:0/2864517778 shutdown_connections 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 -- 172.21.15.110:0/2864517778 wait complete. 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.962+0000 7efd384a7640 1 Processor -- start 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 -- start start 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd30100680 0x7efd30199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 0x7efd3019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd30102230 0x7efd301a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd3010d3b0 con 0x7efd30101870 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd3010d230 con 0x7efd30102230 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd3010d530 con 0x7efd30100680 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 0x7efd3019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 0x7efd3019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60788/0 (socket says 172.21.15.110:60788) 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 -- 172.21.15.110:0/2769939106 learned_addr learned my addr 172.21.15.110:0/2769939106 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:29.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd36a44640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd30102230 0x7efd301a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd36243640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd30100680 0x7efd30199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 -- 172.21.15.110:0/2769939106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd30100680 msgr2=0x7efd30199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd30100680 0x7efd30199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 -- 172.21.15.110:0/2769939106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd30102230 msgr2=0x7efd301a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd30102230 0x7efd301a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 -- 172.21.15.110:0/2769939106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd301a1960 con 0x7efd30101870 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd36243640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd30100680 0x7efd30199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd35a42640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 0x7efd3019a290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efd20005940 tx=0x7efd20004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd277fe640 1 -- 172.21.15.110:0/2769939106 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2003f030 con 0x7efd30101870 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd277fe640 1 -- 172.21.15.110:0/2769939106 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd20003cb0 con 0x7efd30101870 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd301a1bf0 con 0x7efd30101870 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd277fe640 1 -- 172.21.15.110:0/2769939106 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20035070 con 0x7efd30101870 2024-04-25T00:01:29.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.963+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd301a20e0 con 0x7efd30101870 2024-04-25T00:01:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.964+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd301a22b0 con 0x7efd30101870 2024-04-25T00:01:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.965+0000 7efd277fe640 1 -- 172.21.15.110:0/2769939106 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efd20042030 con 0x7efd30101870 2024-04-25T00:01:29.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.966+0000 7efd277fe640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd1c0a2b90 0x7efd1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.966+0000 7efd36243640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd1c0a2b90 0x7efd1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.966+0000 7efd277fe640 1 -- 172.21.15.110:0/2769939106 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efd200efd70 con 0x7efd30101870 2024-04-25T00:01:29.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.966+0000 7efd36243640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd1c0a2b90 0x7efd1c0a5050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efd3019b220 tx=0x7efd2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:29.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:29.969+0000 7efd277fe640 1 -- 172.21.15.110:0/2769939106 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd200bcd00 con 0x7efd30101870 2024-04-25T00:01:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:29 smithi188 ceph-mon[33285]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:01:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:29 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2418643355' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:01:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:29 smithi173 ceph-mon[32402]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:01:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:29 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2418643355' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:01:30.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.143+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd301a2640 con 0x7efd30101870 2024-04-25T00:01:30.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.144+0000 7efd277fe640 1 -- 172.21.15.110:0/2769939106 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7efd200bc600 con 0x7efd30101870 2024-04-25T00:01:30.145 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T00:01:30.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd1c0a2b90 msgr2=0x7efd1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd1c0a2b90 0x7efd1c0a5050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efd3019b220 tx=0x7efd2c008040 comp rx=0 tx=0).stop 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 msgr2=0x7efd3019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 0x7efd3019a290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efd20005940 tx=0x7efd20004430 comp rx=0 tx=0).stop 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 shutdown_connections 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd1c0a2b90 0x7efd1c0a5050 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd30102230 0x7efd301a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd30101870 0x7efd3019a290 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 --2- 172.21.15.110:0/2769939106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd30100680 0x7efd30199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 >> 172.21.15.110:0/2769939106 conn(0x7efd300fbbf0 msgr2=0x7efd30107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 shutdown_connections 2024-04-25T00:01:30.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.146+0000 7efd384a7640 1 -- 172.21.15.110:0/2769939106 wait complete. 2024-04-25T00:01:30.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:01:30.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 -- 172.21.15.110:0/3216754835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4041006a0 msgr2=0x7ff404100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 --2- 172.21.15.110:0/3216754835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4041006a0 0x7ff404100ac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff400009900 tx=0x7ff40002a410 comp rx=0 tx=0).stop 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 -- 172.21.15.110:0/3216754835 shutdown_connections 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 --2- 172.21.15.110:0/3216754835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff404102250 0x7ff404108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 --2- 172.21.15.110:0/3216754835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff404101890 0x7ff404101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 --2- 172.21.15.110:0/3216754835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4041006a0 0x7ff404100ac0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 -- 172.21.15.110:0/3216754835 >> 172.21.15.110:0/3216754835 conn(0x7ff4040fbc10 msgr2=0x7ff4040fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 -- 172.21.15.110:0/3216754835 shutdown_connections 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 -- 172.21.15.110:0/3216754835 wait complete. 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.258+0000 7ff40c34c640 1 Processor -- start 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 -- start start 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4041006a0 0x7ff404199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff404101890 0x7ff40419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff404102250 0x7ff4041a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff40410c220 con 0x7ff404102250 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff40410c0a0 con 0x7ff4041006a0 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff40410c3a0 con 0x7ff404101890 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff404102250 0x7ff4041a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff404102250 0x7ff4041a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60798/0 (socket says 172.21.15.110:60798) 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 -- 172.21.15.110:0/549304809 learned_addr learned my addr 172.21.15.110:0/549304809 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a0e8640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4041006a0 0x7ff404199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff4098e7640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff404101890 0x7ff40419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 -- 172.21.15.110:0/549304809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff404101890 msgr2=0x7ff40419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff404101890 0x7ff40419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 -- 172.21.15.110:0/549304809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4041006a0 msgr2=0x7ff404199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4041006a0 0x7ff404199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 -- 172.21.15.110:0/549304809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4041a1910 con 0x7ff404102250 2024-04-25T00:01:30.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff4098e7640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff404101890 0x7ff40419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:30.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40a8e9640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff404102250 0x7ff4041a1270 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00cad0 tx=0x7ff3fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff3fb7fe640 1 -- 172.21.15.110:0/549304809 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc00ae20 con 0x7ff404102250 2024-04-25T00:01:30.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff3fb7fe640 1 -- 172.21.15.110:0/549304809 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3fc005020 con 0x7ff404102250 2024-04-25T00:01:30.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041a1c00 con 0x7ff404102250 2024-04-25T00:01:30.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff3fb7fe640 1 -- 172.21.15.110:0/549304809 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc00f6b0 con 0x7ff404102250 2024-04-25T00:01:30.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.259+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041a2150 con 0x7ff404102250 2024-04-25T00:01:30.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.261+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4041a23e0 con 0x7ff404102250 2024-04-25T00:01:30.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.262+0000 7ff3fb7fe640 1 -- 172.21.15.110:0/549304809 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff3fc0051d0 con 0x7ff404102250 2024-04-25T00:01:30.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.262+0000 7ff3fb7fe640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff3f00a2c10 0x7ff3f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.262+0000 7ff40a0e8640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff3f00a2c10 0x7ff3f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.263+0000 7ff3fb7fe640 1 -- 172.21.15.110:0/549304809 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff3fc0cf0a0 con 0x7ff404102250 2024-04-25T00:01:30.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.263+0000 7ff40a0e8640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff3f00a2c10 0x7ff3f00a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff40419b1d0 tx=0x7ff400000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.268+0000 7ff3fb7fe640 1 -- 172.21.15.110:0/549304809 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff3fc014030 con 0x7ff404102250 2024-04-25T00:01:30.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.441+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4041a2650 con 0x7ff404102250 2024-04-25T00:01:30.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.441+0000 7ff3fb7fe640 1 -- 172.21.15.110:0/549304809 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7ff3fc09c030 con 0x7ff404102250 2024-04-25T00:01:30.442 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T00:01:30.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff3f00a2c10 msgr2=0x7ff3f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff3f00a2c10 0x7ff3f00a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff40419b1d0 tx=0x7ff400000bc0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff404102250 msgr2=0x7ff4041a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff404102250 0x7ff4041a1270 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00cad0 tx=0x7ff3fc00cf90 comp rx=0 tx=0).stop 2024-04-25T00:01:30.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 shutdown_connections 2024-04-25T00:01:30.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff3f00a2c10 0x7ff3f00a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff404102250 0x7ff4041a1270 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff404101890 0x7ff40419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 --2- 172.21.15.110:0/549304809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4041006a0 0x7ff404199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 >> 172.21.15.110:0/549304809 conn(0x7ff4040fbc10 msgr2=0x7ff404069db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 shutdown_connections 2024-04-25T00:01:30.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.444+0000 7ff40c34c640 1 -- 172.21.15.110:0/549304809 wait complete. 2024-04-25T00:01:30.455 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:01:30.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.556+0000 7f25c9342640 1 -- 172.21.15.110:0/3412894295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c4101890 msgr2=0x7f25c4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.556+0000 7f25c9342640 1 --2- 172.21.15.110:0/3412894295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c4101890 0x7f25c4101d10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009900 tx=0x7f25ac02a410 comp rx=0 tx=0).stop 2024-04-25T00:01:30.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 -- 172.21.15.110:0/3412894295 shutdown_connections 2024-04-25T00:01:30.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 --2- 172.21.15.110:0/3412894295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25c4102250 0x7f25c4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 --2- 172.21.15.110:0/3412894295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c4101890 0x7f25c4101d10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 --2- 172.21.15.110:0/3412894295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c41006a0 0x7f25c4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 -- 172.21.15.110:0/3412894295 >> 172.21.15.110:0/3412894295 conn(0x7f25c40fbc30 msgr2=0x7f25c40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 -- 172.21.15.110:0/3412894295 shutdown_connections 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 -- 172.21.15.110:0/3412894295 wait complete. 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 Processor -- start 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 -- start start 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c41006a0 0x7f25c41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25c4101890 0x7f25c4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c4102250 0x7f25c419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25c410d330 con 0x7f25c41006a0 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f25c410d1b0 con 0x7f25c4102250 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c9342640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f25c410d4b0 con 0x7f25c4101890 2024-04-25T00:01:30.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c41006a0 0x7f25c41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c41006a0 0x7f25c41979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60822/0 (socket says 172.21.15.110:60822) 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.557+0000 7f25c2ffd640 1 -- 172.21.15.110:0/791658027 learned_addr learned my addr 172.21.15.110:0/791658027 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c37fe640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c4102250 0x7f25c419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c2ffd640 1 -- 172.21.15.110:0/791658027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25c4101890 msgr2=0x7f25c4197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c2ffd640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25c4101890 0x7f25c4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c2ffd640 1 -- 172.21.15.110:0/791658027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c4102250 msgr2=0x7f25c419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c2ffd640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c4102250 0x7f25c419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c2ffd640 1 -- 172.21.15.110:0/791658027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c419f5d0 con 0x7f25c41006a0 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c37fe640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c4102250 0x7f25c419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c2ffd640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c41006a0 0x7f25c41979b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c970 tx=0x7f25b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c0ff9640 1 -- 172.21.15.110:0/791658027 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4007c10 con 0x7f25c41006a0 2024-04-25T00:01:30.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c0ff9640 1 -- 172.21.15.110:0/791658027 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25b4007db0 con 0x7f25c41006a0 2024-04-25T00:01:30.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c0ff9640 1 -- 172.21.15.110:0/791658027 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b400f870 con 0x7f25c41006a0 2024-04-25T00:01:30.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c419f8c0 con 0x7f25c41006a0 2024-04-25T00:01:30.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.558+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c419fee0 con 0x7f25c41006a0 2024-04-25T00:01:30.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.559+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c4107a90 con 0x7f25c41006a0 2024-04-25T00:01:30.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.560+0000 7f25c0ff9640 1 -- 172.21.15.110:0/791658027 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f25b400fa10 con 0x7f25c41006a0 2024-04-25T00:01:30.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.561+0000 7f25c0ff9640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f25b00a2c10 0x7f25b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.561+0000 7f25bbfff640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f25b00a2c10 0x7f25b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.561+0000 7f25c0ff9640 1 -- 172.21.15.110:0/791658027 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f25b40cf3c0 con 0x7f25c41006a0 2024-04-25T00:01:30.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.561+0000 7f25bbfff640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f25b00a2c10 0x7f25b00a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25ac02a930 tx=0x7f25ac0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.566+0000 7f25c0ff9640 1 -- 172.21.15.110:0/791658027 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f25b409c3e0 con 0x7f25c41006a0 2024-04-25T00:01:30.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.747+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f25c4062370 con 0x7f25c41006a0 2024-04-25T00:01:30.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.747+0000 7f25c0ff9640 1 -- 172.21.15.110:0/791658027 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f25b409bce0 con 0x7f25c41006a0 2024-04-25T00:01:30.748 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 64 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f25b00a2c10 msgr2=0x7f25b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f25b00a2c10 0x7f25b00a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25ac02a930 tx=0x7f25ac0314a0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c41006a0 msgr2=0x7f25c41979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c41006a0 0x7f25c41979b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c970 tx=0x7f25b400ce30 comp rx=0 tx=0).stop 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 shutdown_connections 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f25b00a2c10 0x7f25b00a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c4102250 0x7f25c419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25c4101890 0x7f25c4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 --2- 172.21.15.110:0/791658027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c41006a0 0x7f25c41979b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 >> 172.21.15.110:0/791658027 conn(0x7f25c40fbc30 msgr2=0x7f25c4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.749+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 shutdown_connections 2024-04-25T00:01:30.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.750+0000 7f25c9342640 1 -- 172.21.15.110:0/791658027 wait complete. 2024-04-25T00:01:30.760 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:30 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2083926267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:30.760 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:30 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2769939106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:30.760 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:30 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/549304809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:30.762 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:30.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 -- 172.21.15.110:0/3198604636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 msgr2=0x7ff270100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 --2- 172.21.15.110:0/3198604636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 0x7ff270100ac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff260009900 tx=0x7ff26002a410 comp rx=0 tx=0).stop 2024-04-25T00:01:30.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 -- 172.21.15.110:0/3198604636 shutdown_connections 2024-04-25T00:01:30.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 --2- 172.21.15.110:0/3198604636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff270102250 0x7ff270108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 --2- 172.21.15.110:0/3198604636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff270101890 0x7ff270101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 --2- 172.21.15.110:0/3198604636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 0x7ff270100ac0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 -- 172.21.15.110:0/3198604636 >> 172.21.15.110:0/3198604636 conn(0x7ff2700fbc30 msgr2=0x7ff2700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 -- 172.21.15.110:0/3198604636 shutdown_connections 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 -- 172.21.15.110:0/3198604636 wait complete. 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 Processor -- start 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.857+0000 7ff2766e1640 1 -- start start 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff2766e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 0x7ff2701979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff2766e1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff270101890 0x7ff270197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff2766e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff270102250 0x7ff27019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff2766e1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff27010c220 con 0x7ff2701006a0 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff2766e1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff27010c0a0 con 0x7ff270101890 2024-04-25T00:01:30.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff2766e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff27010c3a0 con 0x7ff270102250 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 0x7ff2701979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 0x7ff2701979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60846/0 (socket says 172.21.15.110:60846) 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 -- 172.21.15.110:0/1152953240 learned_addr learned my addr 172.21.15.110:0/1152953240 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26f7fe640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff270101890 0x7ff270197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff274c7e640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff270102250 0x7ff27019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 -- 172.21.15.110:0/1152953240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff270102250 msgr2=0x7ff27019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff270102250 0x7ff27019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 -- 172.21.15.110:0/1152953240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff270101890 msgr2=0x7ff270197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff270101890 0x7ff270197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 -- 172.21.15.110:0/1152953240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27019f610 con 0x7ff2701006a0 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff274c7e640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff270102250 0x7ff27019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26f7fe640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff270101890 0x7ff270197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26ffff640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 0x7ff2701979f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff260009230 tx=0x7ff260004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.858+0000 7ff26d7fa640 1 -- 172.21.15.110:0/1152953240 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26003f030 con 0x7ff2701006a0 2024-04-25T00:01:30.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.859+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27019f8a0 con 0x7ff2701006a0 2024-04-25T00:01:30.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.859+0000 7ff26d7fa640 1 -- 172.21.15.110:0/1152953240 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff260003cb0 con 0x7ff2701006a0 2024-04-25T00:01:30.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.859+0000 7ff26d7fa640 1 -- 172.21.15.110:0/1152953240 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260035070 con 0x7ff2701006a0 2024-04-25T00:01:30.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.859+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27019fe60 con 0x7ff2701006a0 2024-04-25T00:01:30.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.860+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2701a0030 con 0x7ff2701006a0 2024-04-25T00:01:30.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.861+0000 7ff26d7fa640 1 -- 172.21.15.110:0/1152953240 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff260042030 con 0x7ff2701006a0 2024-04-25T00:01:30.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.861+0000 7ff26d7fa640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff2580a2b90 0x7ff2580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:30.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.861+0000 7ff26f7fe640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff2580a2b90 0x7ff2580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:30.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.861+0000 7ff26d7fa640 1 -- 172.21.15.110:0/1152953240 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff2600ef750 con 0x7ff2701006a0 2024-04-25T00:01:30.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.862+0000 7ff26f7fe640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff2580a2b90 0x7ff2580a5050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff270198f00 tx=0x7ff264005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:30.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:30.867+0000 7ff26d7fa640 1 -- 172.21.15.110:0/1152953240 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2600bc6e0 con 0x7ff2701006a0 2024-04-25T00:01:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:30 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2083926267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:30 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2769939106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:31.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:30 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/549304809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:31.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.082+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2701a0200 con 0x7ff2701006a0 2024-04-25T00:01:31.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.083+0000 7ff26d7fa640 1 -- 172.21.15.110:0/1152953240 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7ff2600bbfe0 con 0x7ff2701006a0 2024-04-25T00:01:31.083 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:31.083 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:00:02.856564+0000","modified":"2024-04-25T00:00:07.553801+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24457,24476,24512]},"id":1}]} 2024-04-25T00:01:31.083 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-04-25T00:01:31.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff2580a2b90 msgr2=0x7ff2580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff2580a2b90 0x7ff2580a5050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff270198f00 tx=0x7ff264005d90 comp rx=0 tx=0).stop 2024-04-25T00:01:31.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 msgr2=0x7ff2701979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 0x7ff2701979f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff260009230 tx=0x7ff260004340 comp rx=0 tx=0).stop 2024-04-25T00:01:31.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 shutdown_connections 2024-04-25T00:01:31.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff2580a2b90 0x7ff2580a5050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff270102250 0x7ff27019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff270101890 0x7ff270197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 --2- 172.21.15.110:0/1152953240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2701006a0 0x7ff2701979f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 >> 172.21.15.110:0/1152953240 conn(0x7ff2700fbc30 msgr2=0x7ff270069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 shutdown_connections 2024-04-25T00:01:31.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.085+0000 7ff2766e1640 1 -- 172.21.15.110:0/1152953240 wait complete. 2024-04-25T00:01:31.097 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:31.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:30 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2083926267' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:31.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:30 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2769939106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:31.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:30 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/549304809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:31.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 -- 172.21.15.110:0/2675213933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26240ff7f0 msgr2=0x7f26240ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 --2- 172.21.15.110:0/2675213933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26240ff7f0 0x7f26240ffc70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2618009900 tx=0x7f261802a410 comp rx=0 tx=0).stop 2024-04-25T00:01:31.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 -- 172.21.15.110:0/2675213933 shutdown_connections 2024-04-25T00:01:31.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 --2- 172.21.15.110:0/2675213933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26241001b0 0x7f2624109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 --2- 172.21.15.110:0/2675213933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26240ff7f0 0x7f26240ffc70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 --2- 172.21.15.110:0/2675213933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2624102d80 0x7f26241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 -- 172.21.15.110:0/2675213933 >> 172.21.15.110:0/2675213933 conn(0x7f26240fb3b0 msgr2=0x7f26240fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 -- 172.21.15.110:0/2675213933 shutdown_connections 2024-04-25T00:01:31.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.197+0000 7f2628b90640 1 -- 172.21.15.110:0/2675213933 wait complete. 2024-04-25T00:01:31.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2628b90640 1 Processor -- start 2024-04-25T00:01:31.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2628b90640 1 -- start start 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2628b90640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26240ff7f0 0x7f26241979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2628b90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26241001b0 0x7f2624197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2628b90640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2624102d80 0x7f262419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2628b90640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f262410d2e0 con 0x7f2624102d80 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2628b90640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f262410d160 con 0x7f26240ff7f0 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2628b90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f262410d460 con 0x7f26241001b0 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2624102d80 0x7f262419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2624102d80 0x7f262419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60876/0 (socket says 172.21.15.110:60876) 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 -- 172.21.15.110:0/3212355889 learned_addr learned my addr 172.21.15.110:0/3212355889 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2621d9b640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26241001b0 0x7f2624197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f262259c640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26240ff7f0 0x7f26241979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 -- 172.21.15.110:0/3212355889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26241001b0 msgr2=0x7f2624197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26241001b0 0x7f2624197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 -- 172.21.15.110:0/3212355889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26240ff7f0 msgr2=0x7f26241979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26240ff7f0 0x7f26241979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 -- 172.21.15.110:0/3212355889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262419f610 con 0x7f2624102d80 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2621d9b640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26241001b0 0x7f2624197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.198+0000 7f2622d9d640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2624102d80 0x7f262419efa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f261c00eaa0 tx=0x7f261c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.199+0000 7f26137fe640 1 -- 172.21.15.110:0/3212355889 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c00cdb0 con 0x7f2624102d80 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.199+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262419f900 con 0x7f2624102d80 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.199+0000 7f26137fe640 1 -- 172.21.15.110:0/3212355889 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261c005020 con 0x7f2624102d80 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.199+0000 7f26137fe640 1 -- 172.21.15.110:0/3212355889 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0116d0 con 0x7f2624102d80 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.199+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262419fe00 con 0x7f2624102d80 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.200+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2624108d30 con 0x7f2624102d80 2024-04-25T00:01:31.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.201+0000 7f26137fe640 1 -- 172.21.15.110:0/3212355889 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f261c0051d0 con 0x7f2624102d80 2024-04-25T00:01:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.201+0000 7f26137fe640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f260c0a2b90 0x7f260c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.201+0000 7f262259c640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f260c0a2b90 0x7f260c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.201+0000 7f26137fe640 1 -- 172.21.15.110:0/3212355889 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f261c0cf2c0 con 0x7f2624102d80 2024-04-25T00:01:31.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.202+0000 7f262259c640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f260c0a2b90 0x7f260c0a5050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f26140058c0 tx=0x7f261400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.206+0000 7f26137fe640 1 -- 172.21.15.110:0/3212355889 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f261c016030 con 0x7f2624102d80 2024-04-25T00:01:31.383 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.383+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2624062340 con 0x7f2624102d80 2024-04-25T00:01:31.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.385+0000 7f26137fe640 1 -- 172.21.15.110:0/3212355889 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7f261c09c250 con 0x7f2624102d80 2024-04-25T00:01:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:31.385 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:31.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f260c0a2b90 msgr2=0x7f260c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f260c0a2b90 0x7f260c0a5050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f26140058c0 tx=0x7f261400a4c0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2624102d80 msgr2=0x7f262419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2624102d80 0x7f262419efa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f261c00eaa0 tx=0x7f261c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 shutdown_connections 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f260c0a2b90 0x7f260c0a5050 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2624102d80 0x7f262419efa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26241001b0 0x7f2624197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 --2- 172.21.15.110:0/3212355889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26240ff7f0 0x7f26241979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 >> 172.21.15.110:0/3212355889 conn(0x7f26240fb3b0 msgr2=0x7f2624105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 shutdown_connections 2024-04-25T00:01:31.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.387+0000 7f2628b90640 1 -- 172.21.15.110:0/3212355889 wait complete. 2024-04-25T00:01:31.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:01.224042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:02.090769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:31.399 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:31.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 -- 172.21.15.110:0/3878752280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 msgr2=0x7f12f0106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 --2- 172.21.15.110:0/3878752280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 0x7f12f0106f80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009900 tx=0x7f12e002a410 comp rx=0 tx=0).stop 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 -- 172.21.15.110:0/3878752280 shutdown_connections 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 --2- 172.21.15.110:0/3878752280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f01074c0 0x7f12f01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 --2- 172.21.15.110:0/3878752280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 0x7f12f0106f80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 --2- 172.21.15.110:0/3878752280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12f0102260 0x7f12f0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 -- 172.21.15.110:0/3878752280 >> 172.21.15.110:0/3878752280 conn(0x7f12f00fbbf0 msgr2=0x7f12f00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 -- 172.21.15.110:0/3878752280 shutdown_connections 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 -- 172.21.15.110:0/3878752280 wait complete. 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 Processor -- start 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.500+0000 7f12f85a2640 1 -- start start 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f85a2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f0102260 0x7f12f0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f85a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 0x7f12f0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f85a2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12f01074c0 0x7f12f019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f85a2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12f010d2a0 con 0x7f12f0104b90 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f85a2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f12f010d120 con 0x7f12f0102260 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f85a2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f12f010d420 con 0x7f12f01074c0 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 0x7f12f0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 0x7f12f0197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60892/0 (socket says 172.21.15.110:60892) 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 -- 172.21.15.110:0/2527278747 learned_addr learned my addr 172.21.15.110:0/2527278747 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f6b3f640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12f01074c0 0x7f12f019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f633e640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f0102260 0x7f12f0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 -- 172.21.15.110:0/2527278747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12f01074c0 msgr2=0x7f12f019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12f01074c0 0x7f12f019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 -- 172.21.15.110:0/2527278747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f0102260 msgr2=0x7f12f0197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f0102260 0x7f12f0197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 -- 172.21.15.110:0/2527278747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f019f680 con 0x7f12f0104b90 2024-04-25T00:01:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f5b3d640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 0x7f12f0197fa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f12e00314e0 tx=0x7f12e0031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12e77fe640 1 -- 172.21.15.110:0/2527278747 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e003f030 con 0x7f12f0104b90 2024-04-25T00:01:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f019f910 con 0x7f12f0104b90 2024-04-25T00:01:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12e77fe640 1 -- 172.21.15.110:0/2527278747 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12e00044b0 con 0x7f12f0104b90 2024-04-25T00:01:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12e77fe640 1 -- 172.21.15.110:0/2527278747 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0035070 con 0x7f12f0104b90 2024-04-25T00:01:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.501+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f019fe50 con 0x7f12f0104b90 2024-04-25T00:01:31.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.502+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f00fd6e0 con 0x7f12f0104b90 2024-04-25T00:01:31.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.503+0000 7f12e77fe640 1 -- 172.21.15.110:0/2527278747 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f12e0042030 con 0x7f12f0104b90 2024-04-25T00:01:31.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.504+0000 7f12e77fe640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12dc0a2b90 0x7f12dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.504+0000 7f12e77fe640 1 -- 172.21.15.110:0/2527278747 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f12e00ef510 con 0x7f12f0104b90 2024-04-25T00:01:31.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.504+0000 7f12f633e640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12dc0a2b90 0x7f12dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.504+0000 7f12f633e640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12dc0a2b90 0x7f12dc0a5050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009bd0 tx=0x7f12ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.508+0000 7f12e77fe640 1 -- 172.21.15.110:0/2527278747 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12e00bbe50 con 0x7f12f0104b90 2024-04-25T00:01:31.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.685+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f12f0062340 con 0x7f12f0104b90 2024-04-25T00:01:31.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.685+0000 7f12e77fe640 1 -- 172.21.15.110:0/2527278747 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f12e00c0150 con 0x7f12f0104b90 2024-04-25T00:01:31.685 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T00:01:31.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12dc0a2b90 msgr2=0x7f12dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12dc0a2b90 0x7f12dc0a5050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009bd0 tx=0x7f12ec005f90 comp rx=0 tx=0).stop 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 msgr2=0x7f12f0197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 0x7f12f0197fa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f12e00314e0 tx=0x7f12e0031a30 comp rx=0 tx=0).stop 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 shutdown_connections 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12dc0a2b90 0x7f12dc0a5050 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12f01074c0 0x7f12f019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12f0104b90 0x7f12f0197fa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 --2- 172.21.15.110:0/2527278747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f0102260 0x7f12f0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 >> 172.21.15.110:0/2527278747 conn(0x7f12f00fbbf0 msgr2=0x7f12f0102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 shutdown_connections 2024-04-25T00:01:31.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.687+0000 7f12f85a2640 1 -- 172.21.15.110:0/2527278747 wait complete. 2024-04-25T00:01:31.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:01:31.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 -- 172.21.15.110:0/4232525696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 msgr2=0x7f0a0c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 --2- 172.21.15.110:0/4232525696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 0x7f0a0c100ac0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009900 tx=0x7f0a0002a410 comp rx=0 tx=0).stop 2024-04-25T00:01:31.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 -- 172.21.15.110:0/4232525696 shutdown_connections 2024-04-25T00:01:31.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 --2- 172.21.15.110:0/4232525696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a0c102250 0x7f0a0c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 --2- 172.21.15.110:0/4232525696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a0c101890 0x7f0a0c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 --2- 172.21.15.110:0/4232525696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 0x7f0a0c100ac0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 -- 172.21.15.110:0/4232525696 >> 172.21.15.110:0/4232525696 conn(0x7f0a0c0fbc30 msgr2=0x7f0a0c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 -- 172.21.15.110:0/4232525696 shutdown_connections 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 -- 172.21.15.110:0/4232525696 wait complete. 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.799+0000 7f0a1317d640 1 Processor -- start 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1317d640 1 -- start start 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1317d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 0x7f0a0c06a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1317d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a0c101890 0x7f0a0c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1317d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a0c102250 0x7f0a0c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1317d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c10c1a0 con 0x7f0a0c1006a0 2024-04-25T00:01:31.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1317d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c10c020 con 0x7f0a0c102250 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1317d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c10c320 con 0x7f0a0c101890 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 0x7f0a0c06a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 0x7f0a0c06a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60906/0 (socket says 172.21.15.110:60906) 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 -- 172.21.15.110:0/338386286 learned_addr learned my addr 172.21.15.110:0/338386286 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1171a640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a0c102250 0x7f0a0c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a0bfff640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a0c101890 0x7f0a0c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 -- 172.21.15.110:0/338386286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a0c101890 msgr2=0x7f0a0c19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a0c101890 0x7f0a0c19c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 -- 172.21.15.110:0/338386286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a0c102250 msgr2=0x7f0a0c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a0c102250 0x7f0a0c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 -- 172.21.15.110:0/338386286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c19f640 con 0x7f0a0c1006a0 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1171a640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a0c102250 0x7f0a0c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a10f19640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 0x7f0a0c06a8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a000314e0 tx=0x7f0a000042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a09ffb640 1 -- 172.21.15.110:0/338386286 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0003f030 con 0x7f0a0c1006a0 2024-04-25T00:01:31.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c19f870 con 0x7f0a0c1006a0 2024-04-25T00:01:31.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a09ffb640 1 -- 172.21.15.110:0/338386286 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a00003cb0 con 0x7f0a0c1006a0 2024-04-25T00:01:31.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.800+0000 7f0a09ffb640 1 -- 172.21.15.110:0/338386286 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00035070 con 0x7f0a0c1006a0 2024-04-25T00:01:31.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.801+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c19fd60 con 0x7f0a0c1006a0 2024-04-25T00:01:31.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.802+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0c107a90 con 0x7f0a0c1006a0 2024-04-25T00:01:31.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.803+0000 7f0a09ffb640 1 -- 172.21.15.110:0/338386286 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0a00042030 con 0x7f0a0c1006a0 2024-04-25T00:01:31.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.803+0000 7f0a09ffb640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f09fc0a2b90 0x7f09fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:31.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.803+0000 7f0a0bfff640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f09fc0a2b90 0x7f09fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:31.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.803+0000 7f0a09ffb640 1 -- 172.21.15.110:0/338386286 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0a000ef700 con 0x7f0a0c1006a0 2024-04-25T00:01:31.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.804+0000 7f0a0bfff640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f09fc0a2b90 0x7f09fc0a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09f8005e10 tx=0x7f09f8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:31.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.808+0000 7f0a09ffb640 1 -- 172.21.15.110:0/338386286 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0a000bc690 con 0x7f0a0c1006a0 2024-04-25T00:01:31.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.986+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a0c062340 con 0x7f0a0c1006a0 2024-04-25T00:01:31.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.986+0000 7f0a09ffb640 1 -- 172.21.15.110:0/338386286 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f0a000bbf90 con 0x7f0a0c1006a0 2024-04-25T00:01:31.987 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T00:01:31.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f09fc0a2b90 msgr2=0x7f09fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f09fc0a2b90 0x7f09fc0a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09f8005e10 tx=0x7f09f8005da0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 msgr2=0x7f0a0c06a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 0x7f0a0c06a8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a000314e0 tx=0x7f0a000042e0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 shutdown_connections 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f09fc0a2b90 0x7f09fc0a5050 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a0c102250 0x7f0a0c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a0c101890 0x7f0a0c19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 --2- 172.21.15.110:0/338386286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a0c1006a0 0x7f0a0c06a8b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.988+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 >> 172.21.15.110:0/338386286 conn(0x7f0a0c0fbc30 msgr2=0x7f0a0c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.989+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 shutdown_connections 2024-04-25T00:01:31.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:31.989+0000 7f0a1317d640 1 -- 172.21.15.110:0/338386286 wait complete. 2024-04-25T00:01:31.999 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:31 smithi110 ceph-mon[29187]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T00:01:31.999 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:31 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/791658027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:31.999 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:31 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1152953240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:31.999 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:31 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3212355889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:32.000 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:01:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:31 smithi188 ceph-mon[33285]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T00:01:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:31 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/791658027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:31 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1152953240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:31 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3212355889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:32.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 -- 172.21.15.110:0/182638409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74102110 msgr2=0x7faa741089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 --2- 172.21.15.110:0/182638409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74102110 0x7faa741089a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009ca0 tx=0x7faa6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 -- 172.21.15.110:0/182638409 shutdown_connections 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 --2- 172.21.15.110:0/182638409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74102110 0x7faa741089a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 --2- 172.21.15.110:0/182638409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa74101750 0x7faa74101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 --2- 172.21.15.110:0/182638409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74100560 0x7faa74100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 -- 172.21.15.110:0/182638409 >> 172.21.15.110:0/182638409 conn(0x7faa740fbaf0 msgr2=0x7faa740fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 -- 172.21.15.110:0/182638409 shutdown_connections 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.100+0000 7faa7899e640 1 -- 172.21.15.110:0/182638409 wait complete. 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.101+0000 7faa7899e640 1 Processor -- start 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.101+0000 7faa7899e640 1 -- start start 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.101+0000 7faa7899e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74100560 0x7faa74197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.101+0000 7faa7899e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa74101750 0x7faa74197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.101+0000 7faa7899e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74102110 0x7faa7419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.101+0000 7faa7899e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faa7410c0a0 con 0x7faa74100560 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.101+0000 7faa7899e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa7410bf20 con 0x7faa74102110 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.101+0000 7faa7899e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faa7410c220 con 0x7faa74101750 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74100560 0x7faa74197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74100560 0x7faa74197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60936/0 (socket says 172.21.15.110:60936) 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 -- 172.21.15.110:0/738737218 learned_addr learned my addr 172.21.15.110:0/738737218 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa73fff640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74102110 0x7faa7419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa72ffd640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa74101750 0x7faa74197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 -- 172.21.15.110:0/738737218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa74101750 msgr2=0x7faa74197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa74101750 0x7faa74197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 -- 172.21.15.110:0/738737218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74102110 msgr2=0x7faa7419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74102110 0x7faa7419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 -- 172.21.15.110:0/738737218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7419f440 con 0x7faa74100560 2024-04-25T00:01:32.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa72ffd640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa74101750 0x7faa74197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:32.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa737fe640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74100560 0x7faa74197870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faa6400d770 tx=0x7faa6400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa70ff9640 1 -- 172.21.15.110:0/738737218 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64017070 con 0x7faa74100560 2024-04-25T00:01:32.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7419f730 con 0x7faa74100560 2024-04-25T00:01:32.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa70ff9640 1 -- 172.21.15.110:0/738737218 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa64009c30 con 0x7faa74100560 2024-04-25T00:01:32.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa70ff9640 1 -- 172.21.15.110:0/738737218 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64003d00 con 0x7faa74100560 2024-04-25T00:01:32.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.102+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7419fcd0 con 0x7faa74100560 2024-04-25T00:01:32.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.103+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7419fff0 con 0x7faa74100560 2024-04-25T00:01:32.107 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:31 smithi173 ceph-mon[32402]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T00:01:32.107 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:31 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/791658027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:32.107 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:31 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1152953240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:32.107 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:31 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3212355889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:32.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.105+0000 7faa70ff9640 1 -- 172.21.15.110:0/738737218 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7faa64003ea0 con 0x7faa74100560 2024-04-25T00:01:32.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.105+0000 7faa70ff9640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faa5c0a2b90 0x7faa5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.105+0000 7faa72ffd640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faa5c0a2b90 0x7faa5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.105+0000 7faa70ff9640 1 -- 172.21.15.110:0/738737218 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7faa640cedd0 con 0x7faa74100560 2024-04-25T00:01:32.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.106+0000 7faa72ffd640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faa5c0a2b90 0x7faa5c0a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faa74198d30 tx=0x7faa68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.109+0000 7faa70ff9640 1 -- 172.21.15.110:0/738737218 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faa6409bd60 con 0x7faa74100560 2024-04-25T00:01:32.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.283+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faa741a01c0 con 0x7faa74100560 2024-04-25T00:01:32.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.284+0000 7faa70ff9640 1 -- 172.21.15.110:0/738737218 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7faa64009de0 con 0x7faa74100560 2024-04-25T00:01:32.284 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 64 2024-04-25T00:01:32.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faa5c0a2b90 msgr2=0x7faa5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faa5c0a2b90 0x7faa5c0a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faa74198d30 tx=0x7faa68005f90 comp rx=0 tx=0).stop 2024-04-25T00:01:32.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74100560 msgr2=0x7faa74197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74100560 0x7faa74197870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faa6400d770 tx=0x7faa6400dc30 comp rx=0 tx=0).stop 2024-04-25T00:01:32.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 shutdown_connections 2024-04-25T00:01:32.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7faa5c0a2b90 0x7faa5c0a5050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74102110 0x7faa7419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa74101750 0x7faa74197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 --2- 172.21.15.110:0/738737218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa74100560 0x7faa74197870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 >> 172.21.15.110:0/738737218 conn(0x7faa740fbaf0 msgr2=0x7faa740fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 shutdown_connections 2024-04-25T00:01:32.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.286+0000 7faa7899e640 1 -- 172.21.15.110:0/738737218 wait complete. 2024-04-25T00:01:32.297 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 -- 172.21.15.110:0/3307094115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858103340 msgr2=0x7f5858109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 --2- 172.21.15.110:0/3307094115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858103340 0x7f5858109bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5850009ca0 tx=0x7f585002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 -- 172.21.15.110:0/3307094115 shutdown_connections 2024-04-25T00:01:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 --2- 172.21.15.110:0/3307094115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858103340 0x7f5858109bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 --2- 172.21.15.110:0/3307094115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5858102980 0x7f5858102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 --2- 172.21.15.110:0/3307094115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858101790 0x7f5858101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 -- 172.21.15.110:0/3307094115 >> 172.21.15.110:0/3307094115 conn(0x7f58580fcd20 msgr2=0x7f58580ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 -- 172.21.15.110:0/3307094115 shutdown_connections 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.395+0000 7f585e3ce640 1 -- 172.21.15.110:0/3307094115 wait complete. 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f585e3ce640 1 Processor -- start 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f585e3ce640 1 -- start start 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f585e3ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858101790 0x7f5858197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f585e3ce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5858102980 0x7f5858197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f585e3ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858103340 0x7f585819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f585e3ce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f585810d330 con 0x7f5858101790 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f585e3ce640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f585810d1b0 con 0x7f5858102980 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f585e3ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f585810d4b0 con 0x7f5858103340 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f5857fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858101790 0x7f5858197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f5857fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858101790 0x7f5858197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60964/0 (socket says 172.21.15.110:60964) 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.396+0000 7f5857fff640 1 -- 172.21.15.110:0/2205588774 learned_addr learned my addr 172.21.15.110:0/2205588774 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f58577fe640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5858102980 0x7f5858197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f585c96b640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858103340 0x7f585819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f5857fff640 1 -- 172.21.15.110:0/2205588774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858103340 msgr2=0x7f585819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f5857fff640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858103340 0x7f585819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f5857fff640 1 -- 172.21.15.110:0/2205588774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5858102980 msgr2=0x7f5858197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f5857fff640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5858102980 0x7f5858197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f5857fff640 1 -- 172.21.15.110:0/2205588774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585819f590 con 0x7f5858101790 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f5857fff640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858101790 0x7f5858197970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f584000b450 tx=0x7f584000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f58557fa640 1 -- 172.21.15.110:0/2205588774 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840015020 con 0x7f5858101790 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f58557fa640 1 -- 172.21.15.110:0/2205588774 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5840011d40 con 0x7f5858101790 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585819f880 con 0x7f5858101790 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f58557fa640 1 -- 172.21.15.110:0/2205588774 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584000f340 con 0x7f5858101790 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.397+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585819fdd0 con 0x7f5858101790 2024-04-25T00:01:32.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.398+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5858108a50 con 0x7f5858101790 2024-04-25T00:01:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.399+0000 7f58557fa640 1 -- 172.21.15.110:0/2205588774 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5840018020 con 0x7f5858101790 2024-04-25T00:01:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.400+0000 7f58557fa640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f58440a2b90 0x7f58440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.400+0000 7f58577fe640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f58440a2b90 0x7f58440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.400+0000 7f58557fa640 1 -- 172.21.15.110:0/2205588774 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f58400ceb60 con 0x7f5858101790 2024-04-25T00:01:32.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.400+0000 7f58577fe640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f58440a2b90 0x7f58440a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5848005e10 tx=0x7f5848005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.404+0000 7f58557fa640 1 -- 172.21.15.110:0/2205588774 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f584001f020 con 0x7f5858101790 2024-04-25T00:01:32.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.619+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5858062340 con 0x7f5858101790 2024-04-25T00:01:32.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.620+0000 7f58557fa640 1 -- 172.21.15.110:0/2205588774 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7f584000d040 con 0x7f5858101790 2024-04-25T00:01:32.620 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:32.621 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:00:02.856564+0000","modified":"2024-04-25T00:00:07.553801+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24457,24476,24512]},"id":1}]} 2024-04-25T00:01:32.621 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-04-25T00:01:32.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f58440a2b90 msgr2=0x7f58440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f58440a2b90 0x7f58440a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5848005e10 tx=0x7f5848005da0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858101790 msgr2=0x7f5858197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858101790 0x7f5858197970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f584000b450 tx=0x7f584000b910 comp rx=0 tx=0).stop 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 shutdown_connections 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f58440a2b90 0x7f58440a5050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858103340 0x7f585819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5858102980 0x7f5858197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 --2- 172.21.15.110:0/2205588774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5858101790 0x7f5858197970 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 >> 172.21.15.110:0/2205588774 conn(0x7f58580fcd20 msgr2=0x7f5858105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 shutdown_connections 2024-04-25T00:01:32.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.622+0000 7f585e3ce640 1 -- 172.21.15.110:0/2205588774 wait complete. 2024-04-25T00:01:32.634 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:32.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.732+0000 7f79a7a38640 1 -- 172.21.15.110:0/939550074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a0102250 msgr2=0x7f79a0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.732+0000 7f79a7a38640 1 --2- 172.21.15.110:0/939550074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a0102250 0x7f79a0108ae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f799c009900 tx=0x7f799c02a410 comp rx=0 tx=0).stop 2024-04-25T00:01:32.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 -- 172.21.15.110:0/939550074 shutdown_connections 2024-04-25T00:01:32.736 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 --2- 172.21.15.110:0/939550074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a0102250 0x7f79a0108ae0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 --2- 172.21.15.110:0/939550074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a0101890 0x7f79a0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 --2- 172.21.15.110:0/939550074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a01006a0 0x7f79a0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 -- 172.21.15.110:0/939550074 >> 172.21.15.110:0/939550074 conn(0x7f79a00fbc30 msgr2=0x7f79a00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 -- 172.21.15.110:0/939550074 shutdown_connections 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 -- 172.21.15.110:0/939550074 wait complete. 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 Processor -- start 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 -- start start 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a01006a0 0x7f79a0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a0101890 0x7f79a0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a0102250 0x7f79a019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010c230 con 0x7f79a01006a0 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010c0b0 con 0x7f79a0102250 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a7a38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010c3b0 con 0x7f79a0101890 2024-04-25T00:01:32.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a57d4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a01006a0 0x7f79a0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a57d4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a01006a0 0x7f79a0197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60972/0 (socket says 172.21.15.110:60972) 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.733+0000 7f79a57d4640 1 -- 172.21.15.110:0/3875483856 learned_addr learned my addr 172.21.15.110:0/3875483856 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a5fd5640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a0102250 0x7f79a019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a57d4640 1 -- 172.21.15.110:0/3875483856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a0101890 msgr2=0x7f79a0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a57d4640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a0101890 0x7f79a0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a57d4640 1 -- 172.21.15.110:0/3875483856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a0102250 msgr2=0x7f79a019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a57d4640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a0102250 0x7f79a019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a57d4640 1 -- 172.21.15.110:0/3875483856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a019f6a0 con 0x7f79a01006a0 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a5fd5640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a0102250 0x7f79a019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a57d4640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a01006a0 0x7f79a0197a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f799800ccb0 tx=0x7f799800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79967fc640 1 -- 172.21.15.110:0/3875483856 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998003990 con 0x7f79a01006a0 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79967fc640 1 -- 172.21.15.110:0/3875483856 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7998004020 con 0x7f79a01006a0 2024-04-25T00:01:32.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79967fc640 1 -- 172.21.15.110:0/3875483856 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799800f6b0 con 0x7f79a01006a0 2024-04-25T00:01:32.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a019f990 con 0x7f79a01006a0 2024-04-25T00:01:32.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.734+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a019ffb0 con 0x7f79a01006a0 2024-04-25T00:01:32.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.735+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a01a0240 con 0x7f79a01006a0 2024-04-25T00:01:32.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.736+0000 7f79967fc640 1 -- 172.21.15.110:0/3875483856 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7998003b30 con 0x7f79a01006a0 2024-04-25T00:01:32.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.737+0000 7f79967fc640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f798c0a2c10 0x7f798c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:32.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.737+0000 7f79a4fd3640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f798c0a2c10 0x7f798c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:32.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.737+0000 7f79967fc640 1 -- 172.21.15.110:0/3875483856 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f79980cf3e0 con 0x7f79a01006a0 2024-04-25T00:01:32.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.737+0000 7f79a4fd3640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f798c0a2c10 0x7f798c0a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79a0198f90 tx=0x7f7990006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:32.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.741+0000 7f79967fc640 1 -- 172.21.15.110:0/3875483856 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7998014030 con 0x7f79a01006a0 2024-04-25T00:01:32.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.918+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79a01a0410 con 0x7f79a01006a0 2024-04-25T00:01:32.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.919+0000 7f79967fc640 1 -- 172.21.15.110:0/3875483856 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7f799809c370 con 0x7f79a01006a0 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:32.920 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f798c0a2c10 msgr2=0x7f798c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f798c0a2c10 0x7f798c0a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79a0198f90 tx=0x7f7990006c90 comp rx=0 tx=0).stop 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a01006a0 msgr2=0x7f79a0197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a01006a0 0x7f79a0197a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f799800ccb0 tx=0x7f799800a620 comp rx=0 tx=0).stop 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 shutdown_connections 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f798c0a2c10 0x7f798c0a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a0102250 0x7f79a019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79a0101890 0x7f79a0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 --2- 172.21.15.110:0/3875483856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79a01006a0 0x7f79a0197a80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:32.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 >> 172.21.15.110:0/3875483856 conn(0x7f79a00fbc30 msgr2=0x7f79a00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:32.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 shutdown_connections 2024-04-25T00:01:32.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:32.921+0000 7f79a7a38640 1 -- 172.21.15.110:0/3875483856 wait complete. 2024-04-25T00:01:32.931 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:32 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2527278747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:32.932 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:32 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/338386286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:32.932 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:32 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/738737218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:32.932 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:32 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2205588774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:32.933 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:33.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:32 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2527278747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:33.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:32 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/338386286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:33.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:32 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/738737218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:33.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:32 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2205588774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:33.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 -- 172.21.15.110:0/2760165655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 msgr2=0x7fc5ac109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 --2- 172.21.15.110:0/2760165655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 0x7fc5ac109d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009ca0 tx=0x7fc5a402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 -- 172.21.15.110:0/2760165655 shutdown_connections 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 --2- 172.21.15.110:0/2760165655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 0x7fc5ac109d80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 --2- 172.21.15.110:0/2760165655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5ac0ff810 0x7fc5ac0ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 --2- 172.21.15.110:0/2760165655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ac102da0 0x7fc5ac1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 -- 172.21.15.110:0/2760165655 >> 172.21.15.110:0/2760165655 conn(0x7fc5ac0fb3b0 msgr2=0x7fc5ac0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 -- 172.21.15.110:0/2760165655 shutdown_connections 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 -- 172.21.15.110:0/2760165655 wait complete. 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.030+0000 7fc5b214f640 1 Processor -- start 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 -- start start 2024-04-25T00:01:33.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ac0ff810 0x7fc5ac197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 0x7fc5ac197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5ac102da0 0x7fc5ac19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac10d2e0 con 0x7fc5ac1001d0 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac10d160 con 0x7fc5ac0ff810 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac10d460 con 0x7fc5ac102da0 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 0x7fc5ac197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 0x7fc5ac197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60994/0 (socket says 172.21.15.110:60994) 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 -- 172.21.15.110:0/1698925778 learned_addr learned my addr 172.21.15.110:0/1698925778 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5abfff640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5ac102da0 0x7fc5ac19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5ab7fe640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ac0ff810 0x7fc5ac197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 -- 172.21.15.110:0/1698925778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5ac102da0 msgr2=0x7fc5ac19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5ac102da0 0x7fc5ac19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 -- 172.21.15.110:0/1698925778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ac0ff810 msgr2=0x7fc5ac197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ac0ff810 0x7fc5ac197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 -- 172.21.15.110:0/1698925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac19f5b0 con 0x7fc5ac1001d0 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5aaffd640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 0x7fc5ac197ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000d770 tx=0x7fc5a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5a8ff9640 1 -- 172.21.15.110:0/1698925778 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0017070 con 0x7fc5ac1001d0 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5a8ff9640 1 -- 172.21.15.110:0/1698925778 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5a00054d0 con 0x7fc5ac1001d0 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac19f8a0 con 0x7fc5ac1001d0 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5a8ff9640 1 -- 172.21.15.110:0/1698925778 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0004cf0 con 0x7fc5ac1001d0 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.031+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac19fda0 con 0x7fc5ac1001d0 2024-04-25T00:01:33.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.032+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac108c60 con 0x7fc5ac1001d0 2024-04-25T00:01:33.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.034+0000 7fc5a8ff9640 1 -- 172.21.15.110:0/1698925778 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc5a0004e90 con 0x7fc5ac1001d0 2024-04-25T00:01:33.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.034+0000 7fc5a8ff9640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc5940a2b90 0x7fc5940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.034+0000 7fc5ab7fe640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc5940a2b90 0x7fc5940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.034+0000 7fc5a8ff9640 1 -- 172.21.15.110:0/1698925778 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc5a00ceed0 con 0x7fc5ac1001d0 2024-04-25T00:01:33.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.035+0000 7fc5ab7fe640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc5940a2b90 0x7fc5940a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.038+0000 7fc5a8ff9640 1 -- 172.21.15.110:0/1698925778 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc5a009be60 con 0x7fc5ac1001d0 2024-04-25T00:01:33.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:32 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2527278747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:33.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:32 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/338386286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:33.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:32 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/738737218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:33.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:32 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2205588774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:33.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.248+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5ac02d290 con 0x7fc5ac1001d0 2024-04-25T00:01:33.249 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.249+0000 7fc5a8ff9640 1 -- 172.21.15.110:0/1698925778 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7fc5a009b760 con 0x7fc5ac1001d0 2024-04-25T00:01:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:33.249 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:00:02.856564+0000","modified":"2024-04-25T00:00:07.553801+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24457,24476,24512]},"id":1}]} 2024-04-25T00:01:33.249 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-04-25T00:01:33.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc5940a2b90 msgr2=0x7fc5940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc5940a2b90 0x7fc5940a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c005850 comp rx=0 tx=0).stop 2024-04-25T00:01:33.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 msgr2=0x7fc5ac197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 0x7fc5ac197ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000d770 tx=0x7fc5a000dc30 comp rx=0 tx=0).stop 2024-04-25T00:01:33.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 shutdown_connections 2024-04-25T00:01:33.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc5940a2b90 0x7fc5940a5050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5ac102da0 0x7fc5ac19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5ac1001d0 0x7fc5ac197ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 --2- 172.21.15.110:0/1698925778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ac0ff810 0x7fc5ac197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 >> 172.21.15.110:0/1698925778 conn(0x7fc5ac0fb3b0 msgr2=0x7fc5ac105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 shutdown_connections 2024-04-25T00:01:33.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.251+0000 7fc5b214f640 1 -- 172.21.15.110:0/1698925778 wait complete. 2024-04-25T00:01:33.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:00:02.856564+0000', 'modified': '2024-04-25T00:00:07.553801+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24476, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6835/946560580', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 946560580}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 946560580}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/1627462282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 1627462282}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 1627462282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6837/1170231083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 1170231083}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 1170231083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.188:6839/361433277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 361433277}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 361433277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.173:6837/3367670345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 3367670345}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 3367670345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6839/2519479764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 2519479764}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 2519479764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24457, 24476, 24512]} 2024-04-25T00:01:33.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T00:01:33.263 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 -- 172.21.15.110:0/3743354685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98101870 msgr2=0x7efd98101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3743354685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98101870 0x7efd98101cf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efd80009900 tx=0x7efd8002a410 comp rx=0 tx=0).stop 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 -- 172.21.15.110:0/3743354685 shutdown_connections 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3743354685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd98102230 0x7efd98108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3743354685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98101870 0x7efd98101cf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3743354685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd98100680 0x7efd98100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 -- 172.21.15.110:0/3743354685 >> 172.21.15.110:0/3743354685 conn(0x7efd980fbbf0 msgr2=0x7efd980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 -- 172.21.15.110:0/3743354685 shutdown_connections 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.361+0000 7efd9c95a640 1 -- 172.21.15.110:0/3743354685 wait complete. 2024-04-25T00:01:33.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd9c95a640 1 Processor -- start 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd9c95a640 1 -- start start 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd9c95a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd98100680 0x7efd98197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd9c95a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd98101870 0x7efd98197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd9c95a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98102230 0x7efd9819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd9c95a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810d2d0 con 0x7efd98102230 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd9c95a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810d150 con 0x7efd98101870 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd9c95a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810d450 con 0x7efd98100680 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd96d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98102230 0x7efd9819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd96d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98102230 0x7efd9819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32778/0 (socket says 172.21.15.110:32778) 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.362+0000 7efd96d9d640 1 -- 172.21.15.110:0/3924077182 learned_addr learned my addr 172.21.15.110:0/3924077182 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd95d9b640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd98101870 0x7efd98197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd9659c640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd98100680 0x7efd98197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd96d9d640 1 -- 172.21.15.110:0/3924077182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd98100680 msgr2=0x7efd98197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd96d9d640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd98100680 0x7efd98197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd96d9d640 1 -- 172.21.15.110:0/3924077182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd98101870 msgr2=0x7efd98197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd96d9d640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd98101870 0x7efd98197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd96d9d640 1 -- 172.21.15.110:0/3924077182 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9819f620 con 0x7efd98102230 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd9659c640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd98100680 0x7efd98197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd96d9d640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98102230 0x7efd9819efb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd8800cad0 tx=0x7efd8800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd8f7fe640 1 -- 172.21.15.110:0/3924077182 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8800ae20 con 0x7efd98102230 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd8f7fe640 1 -- 172.21.15.110:0/3924077182 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd88005020 con 0x7efd98102230 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd8f7fe640 1 -- 172.21.15.110:0/3924077182 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8800f6b0 con 0x7efd98102230 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd9c95a640 1 -- 172.21.15.110:0/3924077182 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9819f910 con 0x7efd98102230 2024-04-25T00:01:33.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.363+0000 7efd9c95a640 1 -- 172.21.15.110:0/3924077182 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9819fe10 con 0x7efd98102230 2024-04-25T00:01:33.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.365+0000 7efd8f7fe640 1 -- 172.21.15.110:0/3924077182 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efd880051d0 con 0x7efd98102230 2024-04-25T00:01:33.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.365+0000 7efd9f661740 1 Processor -- start 2024-04-25T00:01:33.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd9f661740 1 -- start start 2024-04-25T00:01:33.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd9f661740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e85448860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd9f661740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 0x558e8544d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd9f661740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85450430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd9f661740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558e853abf00 con 0x558e85448440 2024-04-25T00:01:33.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd9f661740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558e85354750 con 0x558e8544dfa0 2024-04-25T00:01:33.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd9f661740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558e853a7ec0 con 0x558e85448e30 2024-04-25T00:01:33.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd8f7fe640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd840a2b90 0x7efd840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e85448860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e85448860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32780/0 (socket says 172.21.15.110:32780) 2024-04-25T00:01:33.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 -- 172.21.15.110:0/4108484068 learned_addr learned my addr 172.21.15.110:0/4108484068 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd96d9d640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd840a2b90 0x7efd840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85450430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd8f7fe640 1 -- 172.21.15.110:0/3924077182 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efd880cf1a0 con 0x7efd98102230 2024-04-25T00:01:33.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd9659c640 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 0x558e8544d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 -- 172.21.15.110:0/4108484068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 msgr2=0x558e8544d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 0x558e8544d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 -- 172.21.15.110:0/4108484068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 msgr2=0x558e85450430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85450430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 -- 172.21.15.110:0/4108484068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e850c02d0 con 0x558e85448440 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd95d9b640 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e85448860 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd80005940 tx=0x7efd800058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e99e6b5998e40d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd96d9d640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd840a2b90 0x7efd840a5050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd8806eac0 tx=0x7efd8806eaf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.366+0000 7efd8cff9640 1 -- 172.21.15.110:0/4108484068 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80035070 con 0x558e85448440 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd8cff9640 1 -- 172.21.15.110:0/4108484068 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd80045070 con 0x558e85448440 2024-04-25T00:01:33.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd8cff9640 1 -- 172.21.15.110:0/4108484068 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80003d80 con 0x558e85448440 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 -- 172.21.15.110:0/4108484068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 msgr2=0x558e85448860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e85448860 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd80005940 tx=0x7efd800058b0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 -- 172.21.15.110:0/4108484068 shutdown_connections 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85450430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 0x558e8544d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 --2- 172.21.15.110:0/4108484068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e85448860 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 -- 172.21.15.110:0/4108484068 >> 172.21.15.110:0/4108484068 conn(0x558e85443ab0 msgr2=0x558e85443eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 -- 172.21.15.110:0/4108484068 shutdown_connections 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.367+0000 7efd9f661740 1 -- 172.21.15.110:0/4108484068 wait complete. 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 1 -- start start 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 1 Processor -- start 2024-04-25T00:01:33.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 0x558e85452810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85452d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85459d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558e85450ef0 con 0x558e85448e30 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558e85367cf0 con 0x558e8544dfa0 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558e85451070 con 0x558e85448440 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9f661740 10 client.0 _ll_register_callbacks cb 0x558e853676e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85452d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85452d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32786/0 (socket says 172.21.15.110:32786) 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 -- 172.21.15.110:0/1813798305 learned_addr learned my addr 172.21.15.110:0/1813798305 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd95d9b640 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85459d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 -- 172.21.15.110:0/1813798305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 msgr2=0x558e85452810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 0x558e85452810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 -- 172.21.15.110:0/1813798305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 msgr2=0x558e85459d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85459d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 -- 172.21.15.110:0/1813798305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e8518a6b0 con 0x558e85448e30 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd95d9b640 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85459d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:33.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd9659c640 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85452d50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd90010840 tx=0x7efd90010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c25514e9b8c783 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.368+0000 7efd7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd7dffb640 1 -- 172.21.15.110:0/1813798305 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90004440 con 0x558e85448e30 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd7dffb640 1 -- 172.21.15.110:0/1813798305 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efd9000e8d0 con 0x558e85448e30 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 1 -- 172.21.15.110:0/1813798305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd80045070 con 0x558e85448e30 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd7dffb640 1 -- 172.21.15.110:0/1813798305 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd90023050 con 0x558e85448e30 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 10 client.15057 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 1 -- 172.21.15.110:0/1813798305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558e8545ce40 con 0x558e85448e30 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd7dffb640 1 -- 172.21.15.110:0/1813798305 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd80045070 con 0x558e85448e30 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd7dffb640 1 -- 172.21.15.110:0/1813798305 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efd9001a070 con 0x558e85448e30 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 10 client.15057 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 20 client.15057 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x558e8545f560 0x558e854609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 4 client.15057 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.369+0000 7efd9f661740 1 -- 172.21.15.110:0/1813798305 --> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e853b9440 con 0x558e8545f560 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.370+0000 7efd95d9b640 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x558e8545f560 0x558e854609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.370+0000 7efd95d9b640 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x558e8545f560 0x558e854609d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d216a18acf383b2b server_cookie=9132e6b504c7afcd in_seq=0 out_seq=0 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.370+0000 7efd7dffb640 10 client.15057 ms_handle_connect on v2:172.21.15.188:6838/361433277 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd7dffb640 1 -- 172.21.15.110:0/1813798305 <== mds.2 v2:172.21.15.188:6838/361433277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e853b9440 con 0x558e8545f560 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd7dffb640 10 client.15057 handle_command_reply: tid=0 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 client.15057 shutdown 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 -- 172.21.15.110:0/1813798305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 msgr2=0x558e85452d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85452d50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd90010840 tx=0x7efd90010d00 comp rx=0 tx=0).stop 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 -- 172.21.15.110:0/1813798305 shutdown_connections 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x558e8545f560 0x558e854609d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85459d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85452d50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd7dffb640 0 client.15057 ms_handle_reset on v2:172.21.15.188:6838/361433277 2024-04-25T00:01:33.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 --2- 172.21.15.110:0/1813798305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 0x558e85452810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 -- 172.21.15.110:0/1813798305 >> 172.21.15.110:0/1813798305 conn(0x558e85443ab0 msgr2=0x558e85442ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 -- 172.21.15.110:0/1813798305 shutdown_connections 2024-04-25T00:01:33.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 1 -- 172.21.15.110:0/1813798305 wait complete. 2024-04-25T00:01:33.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.371+0000 7efd9f661740 20 client.15057 trim_cache size 0 max 16384 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9f661740 1 Processor -- start 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9f661740 1 -- start start 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9f661740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 0x558e85442f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9f661740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85459d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9f661740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85450390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9f661740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558e8545ed10 con 0x558e85448e30 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9f661740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558e8545f3e0 con 0x558e8544dfa0 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9f661740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558e8545fab0 con 0x558e85448440 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85459d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85459d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32800/0 (socket says 172.21.15.110:32800) 2024-04-25T00:01:33.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 -- 172.21.15.110:0/145718878 learned_addr learned my addr 172.21.15.110:0/145718878 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd95d9b640 1 --2- 172.21.15.110:0/145718878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 0x558e85442f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 -- 172.21.15.110:0/145718878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 msgr2=0x558e85442f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 --2- 172.21.15.110:0/145718878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 0x558e85442f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 -- 172.21.15.110:0/145718878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 msgr2=0x558e85450390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 --2- 172.21.15.110:0/145718878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85450390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 -- 172.21.15.110:0/145718878 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e85444780 con 0x558e85448e30 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd9659c640 1 --2- 172.21.15.110:0/145718878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85459d50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd90004870 tx=0x7efd90010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40f769cd62e1ace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd7ffff640 1 -- 172.21.15.110:0/145718878 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9000e720 con 0x558e85448e30 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd7ffff640 1 -- 172.21.15.110:0/145718878 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd90015070 con 0x558e85448e30 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.389+0000 7efd7ffff640 1 -- 172.21.15.110:0/145718878 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90003c20 con 0x558e85448e30 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- 172.21.15.110:0/145718878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 msgr2=0x558e85459d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 --2- 172.21.15.110:0/145718878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85459d50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd90004870 tx=0x7efd90010fa0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- 172.21.15.110:0/145718878 shutdown_connections 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 --2- 172.21.15.110:0/145718878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85450390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 --2- 172.21.15.110:0/145718878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448e30 0x558e85459d50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 --2- 172.21.15.110:0/145718878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448440 0x558e85442f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- 172.21.15.110:0/145718878 >> 172.21.15.110:0/145718878 conn(0x558e85443ab0 msgr2=0x558e8543e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- 172.21.15.110:0/145718878 shutdown_connections 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- 172.21.15.110:0/145718878 wait complete. 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- start start 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 Processor -- start 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e854555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 0x558e85455b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85458b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558e85460850 con 0x558e85448440 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558e85460180 con 0x558e8544dfa0 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.390+0000 7efd9f661740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558e85460f20 con 0x558e85448e30 2024-04-25T00:01:33.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd9f661740 10 client.0 _ll_register_callbacks cb 0x7efd900103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e854555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e854555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32808/0 (socket says 172.21.15.110:32808) 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 -- 172.21.15.110:0/2062960504 learned_addr learned my addr 172.21.15.110:0/2062960504 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd9659c640 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 0x558e85455b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 -- 172.21.15.110:0/2062960504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 msgr2=0x558e85455b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 0x558e85455b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 -- 172.21.15.110:0/2062960504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 msgr2=0x558e85458b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85458b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 -- 172.21.15.110:0/2062960504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e85459110 con 0x558e85448440 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd95d9b640 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e854555c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efd80009900 tx=0x7efd80031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2917e4c11b22a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:33.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd7dffb640 1 -- 172.21.15.110:0/2062960504 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80052020 con 0x558e85448440 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd7dffb640 1 -- 172.21.15.110:0/2062960504 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efd80004230 con 0x558e85448440 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd9f661740 1 -- 172.21.15.110:0/2062960504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd90015070 con 0x558e85448440 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd7dffb640 1 -- 172.21.15.110:0/2062960504 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd80035040 con 0x558e85448440 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd9f661740 10 client.15063 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd9f661740 1 -- 172.21.15.110:0/2062960504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558e854c5690 con 0x558e85448440 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.391+0000 7efd7dffb640 1 -- 172.21.15.110:0/2062960504 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd90015070 con 0x558e85448440 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd7dffb640 1 -- 172.21.15.110:0/2062960504 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efd8004e190 con 0x558e85448440 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 10 client.15063 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 20 client.15063 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x558e854c9dc0 0x558e854c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 4 client.15063 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:01:33.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 1 -- 172.21.15.110:0/2062960504 --> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] -- command(tid 0: {"prefix": "status"}) -- 0x558e853b3790 con 0x558e854c9dc0 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9659c640 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x558e854c9dc0 0x558e854c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9659c640 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x558e854c9dc0 0x558e854c6200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aef1425a86b43e4c server_cookie=d2c390417f5ca01d in_seq=0 out_seq=0 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd7dffb640 10 client.15063 ms_handle_connect on v2:172.21.15.188:6838/361433277 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd7dffb640 1 -- 172.21.15.110:0/2062960504 <== mds.2 v2:172.21.15.188:6838/361433277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558e853b3790 con 0x558e854c9dc0 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd7dffb640 10 client.15063 handle_command_reply: tid=0 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 1 client.15063 shutdown 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 1 -- 172.21.15.110:0/2062960504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 msgr2=0x558e854555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.392+0000 7efd9f661740 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e854555c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efd80009900 tx=0x7efd80031c90 comp rx=0 tx=0).stop 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 1 -- 172.21.15.110:0/2062960504 shutdown_connections 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x558e854c9dc0 0x558e854c6200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e8544dfa0 0x558e85458b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e85448e30 0x558e85455b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 1 --2- 172.21.15.110:0/2062960504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e85448440 0x558e854555c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd7dffb640 0 client.15063 ms_handle_reset on v2:172.21.15.188:6838/361433277 2024-04-25T00:01:33.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 1 -- 172.21.15.110:0/2062960504 >> 172.21.15.110:0/2062960504 conn(0x558e85443ab0 msgr2=0x558e854496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 1 -- 172.21.15.110:0/2062960504 shutdown_connections 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 1 -- 172.21.15.110:0/2062960504 wait complete. 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9f661740 20 client.15063 trim_cache size 0 max 16384 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 2, 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 24457, 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-04-25T00:01:33.400 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 86.843895759000006, 2024-04-25T00:01:33.401 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 21, 2024-04-25T00:01:33.401 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 87, 2024-04-25T00:01:33.401 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T00:01:33.401 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 104.34871698400001 2024-04-25T00:01:33.401 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:01:33.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 -- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd840a2b90 msgr2=0x7efd840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd840a2b90 0x7efd840a5050 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd8806eac0 tx=0x7efd8806eaf0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 -- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98102230 msgr2=0x7efd9819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98102230 0x7efd9819efb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd8800cad0 tx=0x7efd8800cf90 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 -- 172.21.15.110:0/3924077182 shutdown_connections 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd840a2b90 0x7efd840a5050 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd98102230 0x7efd9819efb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd98101870 0x7efd98197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 --2- 172.21.15.110:0/3924077182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd98100680 0x7efd98197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 -- 172.21.15.110:0/3924077182 >> 172.21.15.110:0/3924077182 conn(0x7efd980fbbf0 msgr2=0x7efd98104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 -- 172.21.15.110:0/3924077182 shutdown_connections 2024-04-25T00:01:33.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.393+0000 7efd9c95a640 1 -- 172.21.15.110:0/3924077182 wait complete. 2024-04-25T00:01:33.404 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T00:01:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 -- 172.21.15.110:0/84464840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d81008b0 msgr2=0x7fe2d8100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 --2- 172.21.15.110:0/84464840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d81008b0 0x7fe2d8100d30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009900 tx=0x7fe2cc02a410 comp rx=0 tx=0).stop 2024-04-25T00:01:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 -- 172.21.15.110:0/84464840 shutdown_connections 2024-04-25T00:01:33.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 --2- 172.21.15.110:0/84464840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2d8101270 0x7fe2d8108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 --2- 172.21.15.110:0/84464840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d81008b0 0x7fe2d8100d30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 --2- 172.21.15.110:0/84464840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2d8101c60 0x7fe2d8102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 -- 172.21.15.110:0/84464840 >> 172.21.15.110:0/84464840 conn(0x7fe2d80fb380 msgr2=0x7fe2d80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 -- 172.21.15.110:0/84464840 shutdown_connections 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.503+0000 7fe2dc841640 1 -- 172.21.15.110:0/84464840 wait complete. 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 Processor -- start 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 -- start start 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2d81008b0 0x7fe2d81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d8101270 0x7fe2d8197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2d8101c60 0x7fe2d819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d810c1b0 con 0x7fe2d8101270 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d810c030 con 0x7fe2d81008b0 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d810c330 con 0x7fe2d8101c60 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d8101270 0x7fe2d8197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d8101270 0x7fe2d8197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32820/0 (socket says 172.21.15.110:32820) 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 -- 172.21.15.110:0/2332808719 learned_addr learned my addr 172.21.15.110:0/2332808719 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d659c640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2d81008b0 0x7fe2d81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2d8101c60 0x7fe2d819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 -- 172.21.15.110:0/2332808719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2d8101c60 msgr2=0x7fe2d819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2d8101c60 0x7fe2d819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 -- 172.21.15.110:0/2332808719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2d81008b0 msgr2=0x7fe2d81979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2d81008b0 0x7fe2d81979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 -- 172.21.15.110:0/2332808719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d819f600 con 0x7fe2d8101270 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d8101270 0x7fe2d8197f20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc004770 tx=0x7fe2cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2c77fe640 1 -- 172.21.15.110:0/2332808719 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc03f030 con 0x7fe2d8101270 2024-04-25T00:01:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2c77fe640 1 -- 172.21.15.110:0/2332808719 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2cc003cb0 con 0x7fe2d8101270 2024-04-25T00:01:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.504+0000 7fe2dc841640 1 -- 172.21.15.110:0/2332808719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d819f890 con 0x7fe2d8101270 2024-04-25T00:01:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.505+0000 7fe2c77fe640 1 -- 172.21.15.110:0/2332808719 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc035070 con 0x7fe2d8101270 2024-04-25T00:01:33.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.505+0000 7fe2dc841640 1 -- 172.21.15.110:0/2332808719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d819fdd0 con 0x7fe2d8101270 2024-04-25T00:01:33.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.507+0000 7fe2c77fe640 1 -- 172.21.15.110:0/2332808719 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe2cc042030 con 0x7fe2d8101270 2024-04-25T00:01:33.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.507+0000 7fe2df620740 1 Processor -- start 2024-04-25T00:01:33.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.507+0000 7fe2df620740 1 -- start start 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.507+0000 7fe2df620740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbd9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2df620740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbde930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2df620740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecbe1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2df620740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654ecb3cf00 con 0x5654ecbd9440 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2df620740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654ecae5750 con 0x5654ecbd9e30 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2df620740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654ecb38ec0 con 0x5654ecbdefa0 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2c77fe640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe2c00a2c10 0x7fe2c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbd9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbd9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32822/0 (socket says 172.21.15.110:32822) 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 -- 172.21.15.110:0/1945306854 learned_addr learned my addr 172.21.15.110:0/1945306854 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe2c00a2c10 0x7fe2c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d659c640 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbde930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2c77fe640 1 -- 172.21.15.110:0/2332808719 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe2cc0ef8b0 con 0x7fe2d8101270 2024-04-25T00:01:33.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecbe1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 -- 172.21.15.110:0/1945306854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 msgr2=0x5654ecbe1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecbe1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 -- 172.21.15.110:0/1945306854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 msgr2=0x5654ecbde930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbde930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 -- 172.21.15.110:0/1945306854 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ec8512d0 con 0x5654ecbd9440 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbd9860 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0012ab0 tx=0x7fe2d0012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e579a84b971fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe2c00a2c10 0x7fe2c00a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0010800 tx=0x7fe2d0017000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2c4ff9640 1 -- 172.21.15.110:0/1945306854 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0012590 con 0x5654ecbd9440 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2c4ff9640 1 -- 172.21.15.110:0/1945306854 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2d000ba80 con 0x5654ecbd9440 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.508+0000 7fe2c4ff9640 1 -- 172.21.15.110:0/1945306854 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d000dc30 con 0x5654ecbd9440 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 -- 172.21.15.110:0/1945306854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 msgr2=0x5654ecbd9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbd9860 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0012ab0 tx=0x7fe2d0012f70 comp rx=0 tx=0).stop 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 -- 172.21.15.110:0/1945306854 shutdown_connections 2024-04-25T00:01:33.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecbe1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbde930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 --2- 172.21.15.110:0/1945306854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbd9860 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 -- 172.21.15.110:0/1945306854 >> 172.21.15.110:0/1945306854 conn(0x5654ecbd4ab0 msgr2=0x5654ecbd4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 -- 172.21.15.110:0/1945306854 shutdown_connections 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 -- 172.21.15.110:0/1945306854 wait complete. 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.509+0000 7fe2df620740 1 -- start start 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 1 Processor -- start 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 0x5654ecbe3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbd9e30 0x5654ecbe3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbdefa0 0x5654ecbeae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654ecbe1ef0 con 0x5654ecbdefa0 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654ecaf8cf0 con 0x5654ecbd9440 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654ecbe2070 con 0x5654ecbd9e30 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 10 client.0 _ll_register_callbacks cb 0x5654ecaf86e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:33.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbdefa0 0x5654ecbeae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbdefa0 0x5654ecbeae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32830/0 (socket says 172.21.15.110:32830) 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 -- 172.21.15.110:0/2851717817 learned_addr learned my addr 172.21.15.110:0/2851717817 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d659c640 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 0x5654ecbe3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbd9e30 0x5654ecbe3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 -- 172.21.15.110:0/2851717817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbd9e30 msgr2=0x5654ecbe3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbd9e30 0x5654ecbe3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 -- 172.21.15.110:0/2851717817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 msgr2=0x5654ecbe3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 0x5654ecbe3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 -- 172.21.15.110:0/2851717817 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ec91b6b0 con 0x5654ecbdefa0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbdefa0 0x5654ecbeae10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0013010 tx=0x7fe2d00121c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4751ecf109ef206b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2bdffb640 1 -- 172.21.15.110:0/2851717817 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d000dd40 con 0x5654ecbdefa0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2df620740 1 -- 172.21.15.110:0/2851717817 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2d000ba80 con 0x5654ecbdefa0 2024-04-25T00:01:33.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.510+0000 7fe2bdffb640 1 -- 172.21.15.110:0/2851717817 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe2d0005040 con 0x5654ecbdefa0 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2bdffb640 1 -- 172.21.15.110:0/2851717817 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe2d0005240 con 0x5654ecbdefa0 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2df620740 10 client.15075 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2df620740 1 -- 172.21.15.110:0/2851717817 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5654ecbeddf0 con 0x5654ecbdefa0 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2bdffb640 1 -- 172.21.15.110:0/2851717817 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2d000ba80 con 0x5654ecbdefa0 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2bdffb640 1 -- 172.21.15.110:0/2851717817 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe2d00324d0 con 0x5654ecbdefa0 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2df620740 10 client.15075 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2df620740 10 client.15075 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2df620740 20 client.15075 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2df620740 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x5654ecbf01a0 0x5654ecbf1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2df620740 4 client.15075 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2df620740 1 -- 172.21.15.110:0/2851717817 --> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654ecb4a440 con 0x5654ecbf01a0 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.511+0000 7fe2d659c640 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x5654ecbf01a0 0x5654ecbf1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.512+0000 7fe2bdffb640 10 client.15075 ms_handle_connect on v2:172.21.15.173:6836/3367670345 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.512+0000 7fe2d659c640 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x5654ecbf01a0 0x5654ecbf1610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68cb79ce6a2b2c1 server_cookie=9f2931b39659e8d6 in_seq=0 out_seq=0 2024-04-25T00:01:33.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.512+0000 7fe2bdffb640 1 -- 172.21.15.110:0/2851717817 <== mds.1 v2:172.21.15.173:6836/3367670345 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654ecb4a440 con 0x5654ecbf01a0 2024-04-25T00:01:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.512+0000 7fe2bdffb640 10 client.15075 handle_command_reply: tid=0 2024-04-25T00:01:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.512+0000 7fe2df620740 1 client.15075 shutdown 2024-04-25T00:01:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 -- 172.21.15.110:0/2851717817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbdefa0 msgr2=0x5654ecbeae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbdefa0 0x5654ecbeae10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0013010 tx=0x7fe2d00121c0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 -- 172.21.15.110:0/2851717817 shutdown_connections 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x5654ecbf01a0 0x5654ecbf1610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbdefa0 0x5654ecbeae10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2bdffb640 0 client.15075 ms_handle_reset on v2:172.21.15.173:6836/3367670345 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbd9e30 0x5654ecbe3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 --2- 172.21.15.110:0/2851717817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 0x5654ecbe3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 -- 172.21.15.110:0/2851717817 >> 172.21.15.110:0/2851717817 conn(0x5654ecbd4ab0 msgr2=0x5654ecbd3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 -- 172.21.15.110:0/2851717817 shutdown_connections 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 1 -- 172.21.15.110:0/2851717817 wait complete. 2024-04-25T00:01:33.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.513+0000 7fe2df620740 20 client.15075 trim_cache size 0 max 16384 2024-04-25T00:01:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.530+0000 7fe2df620740 1 Processor -- start 2024-04-25T00:01:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.530+0000 7fe2df620740 1 -- start start 2024-04-25T00:01:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.530+0000 7fe2df620740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbe1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2df620740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbda8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2df620740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecbef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2df620740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654ecbef950 con 0x5654ecbd9440 2024-04-25T00:01:33.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2df620740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654ecbf0020 con 0x5654ecbd9e30 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2df620740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654ecbf06f0 con 0x5654ecbdefa0 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbe1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbe1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32840/0 (socket says 172.21.15.110:32840) 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 -- 172.21.15.110:0/3413472439 learned_addr learned my addr 172.21.15.110:0/3413472439 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecbef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 -- 172.21.15.110:0/3413472439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 msgr2=0x5654ecbef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecbef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbda8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 -- 172.21.15.110:0/3413472439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 msgr2=0x5654ecbda8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbda8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 -- 172.21.15.110:0/3413472439 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ecbd2fc0 con 0x5654ecbd9440 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbda8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2d659c640 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbe1e30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8069850 tx=0x7fe2c8014b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6bb3f94a3e86a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2bffff640 1 -- 172.21.15.110:0/3413472439 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8004d20 con 0x5654ecbd9440 2024-04-25T00:01:33.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2bffff640 1 -- 172.21.15.110:0/3413472439 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c801e8e0 con 0x5654ecbd9440 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.531+0000 7fe2bffff640 1 -- 172.21.15.110:0/3413472439 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8012450 con 0x5654ecbd9440 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- 172.21.15.110:0/3413472439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 msgr2=0x5654ecbe1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbe1e30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8069850 tx=0x7fe2c8014b00 comp rx=0 tx=0).stop 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- 172.21.15.110:0/3413472439 shutdown_connections 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecbef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9e30 0x5654ecbda8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 --2- 172.21.15.110:0/3413472439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9440 0x5654ecbe1e30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- 172.21.15.110:0/3413472439 >> 172.21.15.110:0/3413472439 conn(0x5654ecbd4ab0 msgr2=0x5654ecbd4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- 172.21.15.110:0/3413472439 shutdown_connections 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- 172.21.15.110:0/3413472439 wait complete. 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- start start 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 Processor -- start 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 0x5654ecc54140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9e30 0x5654ecc56690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecc58be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654ecbf1490 con 0x5654ecbd9e30 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654ecbf0dc0 con 0x5654ecbd9440 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654ecbf1b60 con 0x5654ecbdefa0 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2df620740 10 client.0 _ll_register_callbacks cb 0x7fe2d0012880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2d5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9e30 0x5654ecc56690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2d5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9e30 0x5654ecc56690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32852/0 (socket says 172.21.15.110:32852) 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.532+0000 7fe2d5d9b640 1 -- 172.21.15.110:0/1209570735 learned_addr learned my addr 172.21.15.110:0/1209570735 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d5d9b640 1 -- 172.21.15.110:0/1209570735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 msgr2=0x5654ecc58be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecc58be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecc58be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d5d9b640 1 -- 172.21.15.110:0/1209570735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 msgr2=0x5654ecc54140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d659c640 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 0x5654ecc54140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 0x5654ecc54140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d5d9b640 1 -- 172.21.15.110:0/1209570735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ecc59120 con 0x5654ecbd9e30 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d6d9d640 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecc58be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2d5d9b640 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9e30 0x5654ecc56690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc004650 tx=0x7fe2cc0bc080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c8c7c7f6477af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2bdffb640 1 -- 172.21.15.110:0/1209570735 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc049040 con 0x5654ecbd9e30 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2bdffb640 1 -- 172.21.15.110:0/1209570735 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe2cc003800 con 0x5654ecbd9e30 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 1 -- 172.21.15.110:0/1209570735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c801e8e0 con 0x5654ecbd9e30 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2bdffb640 1 -- 172.21.15.110:0/1209570735 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe2cc0c04f0 con 0x5654ecbd9e30 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 10 client.15081 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 1 -- 172.21.15.110:0/1209570735 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5654ecc5a2c0 con 0x5654ecbd9e30 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2bdffb640 1 -- 172.21.15.110:0/1209570735 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c801e8e0 con 0x5654ecbd9e30 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2bdffb640 1 -- 172.21.15.110:0/1209570735 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe2cc0b8860 con 0x5654ecbd9e30 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 10 client.15081 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 20 client.15081 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:33.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x5654ecbf01a0 0x5654ecc5dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 4 client.15081 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.533+0000 7fe2df620740 1 -- 172.21.15.110:0/1209570735 --> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] -- command(tid 0: {"prefix": "status"}) -- 0x5654ecb44790 con 0x5654ecbf01a0 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2d659c640 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x5654ecbf01a0 0x5654ecc5dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2d659c640 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x5654ecbf01a0 0x5654ecc5dc00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=847d75a17f77f8a1 server_cookie=29c237a6f49f3968 in_seq=0 out_seq=0 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2bdffb640 10 client.15081 ms_handle_connect on v2:172.21.15.173:6836/3367670345 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2bdffb640 1 -- 172.21.15.110:0/1209570735 <== mds.1 v2:172.21.15.173:6836/3367670345 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5654ecb44790 con 0x5654ecbf01a0 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2bdffb640 10 client.15081 handle_command_reply: tid=0 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 client.15081 shutdown 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 -- 172.21.15.110:0/1209570735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9e30 msgr2=0x5654ecc56690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9e30 0x5654ecc56690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc004650 tx=0x7fe2cc0bc080 comp rx=0 tx=0).stop 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 -- 172.21.15.110:0/1209570735 shutdown_connections 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x5654ecbf01a0 0x5654ecc5dc00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654ecbdefa0 0x5654ecc58be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2bdffb640 0 client.15081 ms_handle_reset on v2:172.21.15.173:6836/3367670345 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654ecbd9e30 0x5654ecc56690 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 --2- 172.21.15.110:0/1209570735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654ecbd9440 0x5654ecc54140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.534+0000 7fe2df620740 1 -- 172.21.15.110:0/1209570735 >> 172.21.15.110:0/1209570735 conn(0x5654ecbd4ab0 msgr2=0x5654ecbe2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2df620740 1 -- 172.21.15.110:0/1209570735 shutdown_connections 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2df620740 1 -- 172.21.15.110:0/1209570735 wait complete. 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2df620740 20 client.15081 trim_cache size 0 max 16384 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 1, 2024-04-25T00:01:33.542 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 24476, 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 87.990863653000005, 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 21, 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 87, 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 107.307772962 2024-04-25T00:01:33.543 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:01:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 -- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe2c00a2c10 msgr2=0x7fe2c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe2c00a2c10 0x7fe2c00a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0010800 tx=0x7fe2d0017000 comp rx=0 tx=0).stop 2024-04-25T00:01:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 -- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d8101270 msgr2=0x7fe2d8197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d8101270 0x7fe2d8197f20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc004770 tx=0x7fe2cc0042f0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 -- 172.21.15.110:0/2332808719 shutdown_connections 2024-04-25T00:01:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe2c00a2c10 0x7fe2c00a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2d8101c60 0x7fe2d819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d8101270 0x7fe2d8197f20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 --2- 172.21.15.110:0/2332808719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2d81008b0 0x7fe2d81979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 -- 172.21.15.110:0/2332808719 >> 172.21.15.110:0/2332808719 conn(0x7fe2d80fb380 msgr2=0x7fe2d80ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 -- 172.21.15.110:0/2332808719 shutdown_connections 2024-04-25T00:01:33.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.535+0000 7fe2dc841640 1 -- 172.21.15.110:0/2332808719 wait complete. 2024-04-25T00:01:33.546 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.644+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1524678692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c102230 msgr2=0x7f6c8c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.644+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1524678692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c102230 0x7f6c8c108ac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009900 tx=0x7f6c8402a410 comp rx=0 tx=0).stop 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1524678692 shutdown_connections 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1524678692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c102230 0x7f6c8c108ac0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1524678692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c8c101870 0x7f6c8c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1524678692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c8c100680 0x7f6c8c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1524678692 >> 172.21.15.110:0/1524678692 conn(0x7f6c8c0fbbf0 msgr2=0x7f6c8c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1524678692 shutdown_connections 2024-04-25T00:01:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1524678692 wait complete. 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 Processor -- start 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 -- start start 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c8c100680 0x7f6c8c1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c101870 0x7f6c8c196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c8c102230 0x7f6c8c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10c190 con 0x7f6c8c101870 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10c010 con 0x7f6c8c100680 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.645+0000 7f6c91c4d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10c310 con 0x7f6c8c102230 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c101870 0x7f6c8c196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c101870 0x7f6c8c196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32870/0 (socket says 172.21.15.110:32870) 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 -- 172.21.15.110:0/1624589874 learned_addr learned my addr 172.21.15.110:0/1624589874 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c8c100680 0x7f6c8c1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8bfff640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c8c102230 0x7f6c8c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 -- 172.21.15.110:0/1624589874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c8c102230 msgr2=0x7f6c8c19dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c8c102230 0x7f6c8c19dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 -- 172.21.15.110:0/1624589874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c8c100680 msgr2=0x7f6c8c1967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c8c100680 0x7f6c8c1967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 -- 172.21.15.110:0/1624589874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c19e400 con 0x7f6c8c101870 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c8affd640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c101870 0x7f6c8c196d20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6c80007b50 tx=0x7f6c8000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c88ff9640 1 -- 172.21.15.110:0/1624589874 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80007c40 con 0x7f6c8c101870 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1624589874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c068930 con 0x7f6c8c101870 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c88ff9640 1 -- 172.21.15.110:0/1624589874 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c80007de0 con 0x7f6c8c101870 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c88ff9640 1 -- 172.21.15.110:0/1624589874 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000ad40 con 0x7f6c8c101870 2024-04-25T00:01:33.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.646+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1624589874 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c068ed0 con 0x7f6c8c101870 2024-04-25T00:01:33.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.648+0000 7f6c88ff9640 1 -- 172.21.15.110:0/1624589874 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6c8000eb30 con 0x7f6c8c101870 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c94a3a740 1 Processor -- start 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c94a3a740 1 -- start start 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c88ff9640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6c740a2b90 0x7f6c740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 0x55a7b107f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1084930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 0x55a7b1087430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a7b0fe2f00 con 0x55a7b107fe30 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a7b0f8b750 con 0x55a7b107f440 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a7b0fdeec0 con 0x55a7b1084fa0 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8bfff640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6c740a2b90 0x7f6c740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c88ff9640 1 -- 172.21.15.110:0/1624589874 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6c800d05c0 con 0x7f6c8c101870 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1084930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1084930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32882/0 (socket says 172.21.15.110:32882) 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 -- 172.21.15.110:0/2574311115 learned_addr learned my addr 172.21.15.110:0/2574311115 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/2574311115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 0x55a7b107f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 -- 172.21.15.110:0/2574311115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 msgr2=0x55a7b1087430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 --2- 172.21.15.110:0/2574311115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 0x55a7b1087430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 -- 172.21.15.110:0/2574311115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 msgr2=0x55a7b107f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 --2- 172.21.15.110:0/2574311115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 0x55a7b107f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 -- 172.21.15.110:0/2574311115 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b0cf72d0 con 0x55a7b107fe30 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8bfff640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6c740a2b90 0x7f6c740a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6c840314e0 tx=0x7f6c84032c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.649+0000 7f6c8affd640 1 --2- 172.21.15.110:0/2574311115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1084930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c0a0 tx=0x7f6c80099240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fdf8845f980412 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c71ffb640 1 -- 172.21.15.110:0/2574311115 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8004b820 con 0x55a7b107fe30 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c71ffb640 1 -- 172.21.15.110:0/2574311115 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c8000a440 con 0x55a7b107fe30 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c71ffb640 1 -- 172.21.15.110:0/2574311115 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8004b640 con 0x55a7b107fe30 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2574311115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 msgr2=0x55a7b1084930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2574311115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1084930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c0a0 tx=0x7f6c80099240 comp rx=0 tx=0).stop 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2574311115 shutdown_connections 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2574311115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 0x55a7b1087430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2574311115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1084930 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2574311115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 0x55a7b107f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2574311115 >> 172.21.15.110:0/2574311115 conn(0x55a7b107aab0 msgr2=0x55a7b107aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2574311115 shutdown_connections 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.650+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2574311115 wait complete. 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 1 -- start start 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 1 Processor -- start 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107f440 0x55a7b1089870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1089db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b1084fa0 0x55a7b1090df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1087ef0 con 0x55a7b107fe30 2024-04-25T00:01:33.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a7b0f9ecf0 con 0x55a7b1084fa0 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1088070 con 0x55a7b107f440 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c94a3a740 10 client.0 _ll_register_callbacks cb 0x55a7b0f9e6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1089db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1089db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32888/0 (socket says 172.21.15.110:32888) 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8affd640 1 -- 172.21.15.110:0/2080094741 learned_addr learned my addr 172.21.15.110:0/2080094741 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8bfff640 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b1084fa0 0x55a7b1090df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107f440 0x55a7b1089870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8affd640 1 -- 172.21.15.110:0/2080094741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107f440 msgr2=0x55a7b1089870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8affd640 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107f440 0x55a7b1089870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8affd640 1 -- 172.21.15.110:0/2080094741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b1084fa0 msgr2=0x55a7b1090df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8affd640 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b1084fa0 0x55a7b1090df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.651+0000 7f6c8affd640 1 -- 172.21.15.110:0/2080094741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b0dc16b0 con 0x55a7b107fe30 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c8affd640 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1089db0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6c800a5590 tx=0x7f6c800a1300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defd4c5193497aed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c6affd640 1 -- 172.21.15.110:0/2080094741 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80051020 con 0x55a7b107fe30 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c6affd640 1 -- 172.21.15.110:0/2080094741 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6c800a5830 con 0x55a7b107fe30 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2080094741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7b0ff0440 con 0x55a7b107fe30 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c6affd640 1 -- 172.21.15.110:0/2080094741 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c8009d5e0 con 0x55a7b107fe30 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 10 client.15093 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2080094741 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a7b1093dd0 con 0x55a7b107fe30 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c6affd640 1 -- 172.21.15.110:0/2080094741 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a7b0ff0440 con 0x55a7b107fe30 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c6affd640 1 -- 172.21.15.110:0/2080094741 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6c80014070 con 0x55a7b107fe30 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 10 client.15093 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 20 client.15093 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a7b1095120 0x55a7b1097320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 4 client.15093 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.652+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2080094741 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7b0fea790 con 0x55a7b1095120 2024-04-25T00:01:33.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.653+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a7b1095120 0x55a7b1097320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.653+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a7b1095120 0x55a7b1097320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ece8f633b18e2c server_cookie=d39083461af62dbc in_seq=0 out_seq=0 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.653+0000 7f6c6affd640 10 client.15093 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c6affd640 1 -- 172.21.15.110:0/2080094741 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7b0fea790 con 0x55a7b1095120 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c6affd640 10 client.15093 handle_command_reply: tid=0 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 client.15093 shutdown 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2080094741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 msgr2=0x55a7b1089db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1089db0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6c800a5590 tx=0x7f6c800a1300 comp rx=0 tx=0).stop 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2080094741 shutdown_connections 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a7b1095120 0x55a7b1097320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b1084fa0 0x55a7b1090df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107fe30 0x55a7b1089db0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c6affd640 0 client.15093 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/2080094741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107f440 0x55a7b1089870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2080094741 >> 172.21.15.110:0/2080094741 conn(0x55a7b107aab0 msgr2=0x55a7b1079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2080094741 shutdown_connections 2024-04-25T00:01:33.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 1 -- 172.21.15.110:0/2080094741 wait complete. 2024-04-25T00:01:33.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.654+0000 7f6c94a3a740 20 client.15093 trim_cache size 0 max 16384 2024-04-25T00:01:33.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.672+0000 7f6c94a3a740 1 Processor -- start 2024-04-25T00:01:33.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.672+0000 7f6c94a3a740 1 -- start start 2024-04-25T00:01:33.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107f440 0x55a7b107d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107fe30 0x55a7b1090db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 0x55a7b107df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1095660 con 0x55a7b107f440 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1095d30 con 0x55a7b107fe30 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1096400 con 0x55a7b1084fa0 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107f440 0x55a7b107d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107f440 0x55a7b107d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32898/0 (socket says 172.21.15.110:32898) 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 -- 172.21.15.110:0/3560185683 learned_addr learned my addr 172.21.15.110:0/3560185683 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8bfff640 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 0x55a7b107df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8affd640 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107fe30 0x55a7b1090db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 -- 172.21.15.110:0/3560185683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 msgr2=0x55a7b107df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 0x55a7b107df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 -- 172.21.15.110:0/3560185683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107fe30 msgr2=0x55a7b1090db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107fe30 0x55a7b1090db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 -- 172.21.15.110:0/3560185683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b107a8e0 con 0x55a7b107f440 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8affd640 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107fe30 0x55a7b1090db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107f440 0x55a7b107d9c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c000f80 tx=0x7f6c7c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7763bffa8e14709e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c717fa640 1 -- 172.21.15.110:0/3560185683 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c024070 con 0x55a7b107f440 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c717fa640 1 -- 172.21.15.110:0/3560185683 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c7c01f6e0 con 0x55a7b107f440 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.673+0000 7f6c717fa640 1 -- 172.21.15.110:0/3560185683 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c01dbf0 con 0x55a7b107f440 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 -- 172.21.15.110:0/3560185683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107f440 msgr2=0x55a7b107d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107f440 0x55a7b107d9c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c000f80 tx=0x7f6c7c009500 comp rx=0 tx=0).stop 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 -- 172.21.15.110:0/3560185683 shutdown_connections 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b1084fa0 0x55a7b107df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107fe30 0x55a7b1090db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/3560185683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b107f440 0x55a7b107d9c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 -- 172.21.15.110:0/3560185683 >> 172.21.15.110:0/3560185683 conn(0x55a7b107aab0 msgr2=0x55a7b10812d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 -- 172.21.15.110:0/3560185683 shutdown_connections 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 -- 172.21.15.110:0/3560185683 wait complete. 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.674+0000 7f6c94a3a740 1 -- start start 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 1 Processor -- start 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 0x55a7b108eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107fe30 0x55a7b108f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b1084fa0 0x55a7b108fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a7b10971a0 con 0x55a7b1084fa0 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1096ad0 con 0x55a7b107f440 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a7b1097870 con 0x55a7b107fe30 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 10 client.0 _ll_register_callbacks cb 0x7f6c8000aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b1084fa0 0x55a7b108fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b1084fa0 0x55a7b108fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32902/0 (socket says 172.21.15.110:32902) 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 -- 172.21.15.110:0/1215795258 learned_addr learned my addr 172.21.15.110:0/1215795258 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 0x55a7b108eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8affd640 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107fe30 0x55a7b108f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 -- 172.21.15.110:0/1215795258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107fe30 msgr2=0x55a7b108f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107fe30 0x55a7b108f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 -- 172.21.15.110:0/1215795258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 msgr2=0x55a7b108eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 0x55a7b108eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 -- 172.21.15.110:0/1215795258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b1090250 con 0x55a7b1084fa0 2024-04-25T00:01:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c8bfff640 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b1084fa0 0x55a7b108fa70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6c84033000 tx=0x7f6c84009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d845cdd03c9b7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:33.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c6affd640 1 -- 172.21.15.110:0/1215795258 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84035050 con 0x55a7b1084fa0 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c6affd640 1 -- 172.21.15.110:0/1215795258 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6c84005450 con 0x55a7b1084fa0 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 1 -- 172.21.15.110:0/1215795258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c7c01f6e0 con 0x55a7b1084fa0 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c6affd640 1 -- 172.21.15.110:0/1215795258 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c8405a070 con 0x55a7b1084fa0 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.675+0000 7f6c94a3a740 10 client.15099 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c94a3a740 1 -- 172.21.15.110:0/1215795258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a7b10fc0e0 con 0x55a7b1084fa0 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c6affd640 1 -- 172.21.15.110:0/1215795258 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c7c01f6e0 con 0x55a7b1084fa0 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c6affd640 1 -- 172.21.15.110:0/1215795258 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6c84003820 con 0x55a7b1084fa0 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c94a3a740 10 client.15099 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c94a3a740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c94a3a740 20 client.15099 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a7b1100d70 0x55a7b10ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:33.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c94a3a740 4 client.15099 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c94a3a740 1 -- 172.21.15.110:0/1215795258 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "status"}) -- 0x55a7b0f68b80 con 0x55a7b1100d70 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a7b1100d70 0x55a7b10ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c6affd640 10 client.15099 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.676+0000 7f6c8b7fe640 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a7b1100d70 0x55a7b10ffa80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7da65f1277c5ab server_cookie=fb4f3b70b055d595 in_seq=0 out_seq=0 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c6affd640 1 -- 172.21.15.110:0/1215795258 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a7b0f68b80 con 0x55a7b1100d70 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c6affd640 10 client.15099 handle_command_reply: tid=0 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 client.15099 shutdown 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 -- 172.21.15.110:0/1215795258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b1084fa0 msgr2=0x55a7b108fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b1084fa0 0x55a7b108fa70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6c84033000 tx=0x7f6c84009260 comp rx=0 tx=0).stop 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 -- 172.21.15.110:0/1215795258 shutdown_connections 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a7b1100d70 0x55a7b10ffa80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a7b1084fa0 0x55a7b108fa70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c6affd640 0 client.15099 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7b107fe30 0x55a7b108f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 --2- 172.21.15.110:0/1215795258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7b107f440 0x55a7b108eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 -- 172.21.15.110:0/1215795258 >> 172.21.15.110:0/1215795258 conn(0x55a7b107aab0 msgr2=0x55a7b107d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 -- 172.21.15.110:0/1215795258 shutdown_connections 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 1 -- 172.21.15.110:0/1215795258 wait complete. 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c94a3a740 20 client.15099 trim_cache size 0 max 16384 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:01:33.685 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 24512, 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 90.157621852000005, 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 21, 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 87, 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 95.861272726999999 2024-04-25T00:01:33.686 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:01:33.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6c740a2b90 msgr2=0x7f6c740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6c740a2b90 0x7f6c740a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6c840314e0 tx=0x7f6c84032c60 comp rx=0 tx=0).stop 2024-04-25T00:01:33.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c101870 msgr2=0x7f6c8c196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:33.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c101870 0x7f6c8c196d20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6c80007b50 tx=0x7f6c8000e420 comp rx=0 tx=0).stop 2024-04-25T00:01:33.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1624589874 shutdown_connections 2024-04-25T00:01:33.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6c740a2b90 0x7f6c740a5050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c8c102230 0x7f6c8c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c8c101870 0x7f6c8c196d20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 --2- 172.21.15.110:0/1624589874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c8c100680 0x7f6c8c1967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:33.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.677+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1624589874 >> 172.21.15.110:0/1624589874 conn(0x7f6c8c0fbbf0 msgr2=0x7f6c8c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:33.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.678+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1624589874 shutdown_connections 2024-04-25T00:01:33.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:33.678+0000 7f6c91c4d640 1 -- 172.21.15.110:0/1624589874 wait complete. 2024-04-25T00:01:33.689 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T00:01:33.689 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:01:33.707 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T00:01:33.708 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T00:01:33.943 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:33 smithi188 ceph-mon[33285]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T00:01:34.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:33 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3875483856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:34.057 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:33 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1698925778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:34.058 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:33 smithi110 ceph-mon[29187]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T00:01:34.058 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:33 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3875483856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:34.058 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:33 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1698925778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:33 smithi173 ceph-mon[32402]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:33 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3875483856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:33 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1698925778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:35.704 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:35 smithi110 ceph-mon[29187]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:01:35.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.804+0000 7f1f360fe700 1 -- 172.21.15.110:0/3896220039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30102d20 msgr2=0x7f1f301095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.804+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3896220039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30102d20 0x7f1f301095d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800b0d0 tx=0x7f1f2800b3e0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.804+0000 7f1f360fe700 1 -- 172.21.15.110:0/3896220039 shutdown_connections 2024-04-25T00:01:35.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.804+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3896220039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30102d20 0x7f1f301095d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.804+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3896220039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f30101180 0x7f1f301015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.804+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3896220039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f30102380 0x7f1f301027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 -- 172.21.15.110:0/3896220039 >> 172.21.15.110:0/3896220039 conn(0x7f1f300fc720 msgr2=0x7f1f300feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 -- 172.21.15.110:0/3896220039 shutdown_connections 2024-04-25T00:01:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 -- 172.21.15.110:0/3896220039 wait complete. 2024-04-25T00:01:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 Processor -- start 2024-04-25T00:01:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 -- start start 2024-04-25T00:01:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30101180 0x7f1f301050c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f30102380 0x7f1f30103760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f30102d20 0x7f1f30103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010cd10 con 0x7f1f30101180 2024-04-25T00:01:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010cb90 con 0x7f1f30102d20 2024-04-25T00:01:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.805+0000 7f1f360fe700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010ce90 con 0x7f1f30102380 2024-04-25T00:01:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30101180 0x7f1f301050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30101180 0x7f1f301050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32920/0 (socket says 172.21.15.110:32920) 2024-04-25T00:01:35.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 -- 172.21.15.110:0/3435289619 learned_addr learned my addr 172.21.15.110:0/3435289619 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2effd700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f30102380 0x7f1f30103760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2ffff700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f30102d20 0x7f1f30103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 -- 172.21.15.110:0/3435289619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f30102380 msgr2=0x7f1f30103760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f30102380 0x7f1f30103760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 -- 172.21.15.110:0/3435289619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f30102d20 msgr2=0x7f1f30103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f30102d20 0x7f1f30103ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:35.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 -- 172.21.15.110:0/3435289619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f30104570 con 0x7f1f30101180 2024-04-25T00:01:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2effd700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f30102380 0x7f1f30103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2f7fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30101180 0x7f1f301050c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000b500 tx=0x7f1f2000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2cff9700 1 -- 172.21.15.110:0/3435289619 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20015020 con 0x7f1f30101180 2024-04-25T00:01:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2cff9700 1 -- 172.21.15.110:0/3435289619 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f20011ce0 con 0x7f1f30101180 2024-04-25T00:01:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f301a2650 con 0x7f1f30101180 2024-04-25T00:01:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f2cff9700 1 -- 172.21.15.110:0/3435289619 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20010d60 con 0x7f1f30101180 2024-04-25T00:01:35.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.806+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f301a2be0 con 0x7f1f30101180 2024-04-25T00:01:35.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.807+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f30068f20 con 0x7f1f30101180 2024-04-25T00:01:35.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.808+0000 7f1f2cff9700 1 -- 172.21.15.110:0/3435289619 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1f20018020 con 0x7f1f30101180 2024-04-25T00:01:35.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.809+0000 7f1f2cff9700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f180a2ca0 0x7f1f180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:35.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.809+0000 7f1f2effd700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f180a2ca0 0x7f1f180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:35.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.809+0000 7f1f2cff9700 1 -- 172.21.15.110:0/3435289619 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1f200cedd0 con 0x7f1f30101180 2024-04-25T00:01:35.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.809+0000 7f1f2effd700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f180a2ca0 0x7f1f180a5160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1f240099d0 tx=0x7f1f24008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:35.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:35.812+0000 7f1f2cff9700 1 -- 172.21.15.110:0/3435289619 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f2001f020 con 0x7f1f30101180 2024-04-25T00:01:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:35 smithi188 ceph-mon[33285]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:01:36.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.028+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1f300fe930 con 0x7f1f180a2ca0 2024-04-25T00:01:36.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.029+0000 7f1f2cff9700 1 -- 172.21.15.110:0/3435289619 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1f300fe930 con 0x7f1f180a2ca0 2024-04-25T00:01:36.029 INFO:teuthology.orchestra.run.smithi110.stdout:Backend: cephadm 2024-04-25T00:01:36.029 INFO:teuthology.orchestra.run.smithi110.stdout:Available: Yes 2024-04-25T00:01:36.029 INFO:teuthology.orchestra.run.smithi110.stdout:Paused: No 2024-04-25T00:01:36.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f180a2ca0 msgr2=0x7f1f180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f180a2ca0 0x7f1f180a5160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1f240099d0 tx=0x7f1f24008040 comp rx=0 tx=0).stop 2024-04-25T00:01:36.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30101180 msgr2=0x7f1f301050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:36.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30101180 0x7f1f301050c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000b500 tx=0x7f1f2000b8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 shutdown_connections 2024-04-25T00:01:36.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f180a2ca0 0x7f1f180a5160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f30102d20 0x7f1f30103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30101180 0x7f1f301050c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 --2- 172.21.15.110:0/3435289619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f30102380 0x7f1f30103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:36.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 >> 172.21.15.110:0/3435289619 conn(0x7f1f300fc720 msgr2=0x7f1f300fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:36.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 shutdown_connections 2024-04-25T00:01:36.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:36.031+0000 7f1f360fe700 1 -- 172.21.15.110:0/3435289619 wait complete. 2024-04-25T00:01:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:35 smithi173 ceph-mon[32402]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:01:36.903 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T00:01:37.140 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:37 smithi188 ceph-mon[33285]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:37 smithi188 ceph-mon[33285]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T00:01:38.018 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:37 smithi110 ceph-mon[29187]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:38.018 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:37 smithi110 ceph-mon[29187]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T00:01:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:37 smithi173 ceph-mon[32402]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:37 smithi173 ceph-mon[32402]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 -- 172.21.15.110:0/990283967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 msgr2=0x7fed741026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 --2- 172.21.15.110:0/990283967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 0x7fed741026b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fed680099c0 tx=0x7fed68009cd0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 -- 172.21.15.110:0/990283967 shutdown_connections 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 --2- 172.21.15.110:0/990283967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed74103e30 0x7fed7410a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 --2- 172.21.15.110:0/990283967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 0x7fed741026b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 --2- 172.21.15.110:0/990283967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed74103490 0x7fed741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 -- 172.21.15.110:0/990283967 >> 172.21.15.110:0/990283967 conn(0x7fed740fd830 msgr2=0x7fed740ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 -- 172.21.15.110:0/990283967 shutdown_connections 2024-04-25T00:01:38.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.965+0000 7fed7abd6700 1 -- 172.21.15.110:0/990283967 wait complete. 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed7abd6700 1 Processor -- start 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed7abd6700 1 -- start start 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed7abd6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 0x7fed74198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed7abd6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed74103490 0x7fed74198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed7abd6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed74103e30 0x7fed7419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed7abd6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410dd80 con 0x7fed74102290 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed7abd6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410dc00 con 0x7fed74103490 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed7abd6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410df00 con 0x7fed74103e30 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed78972700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 0x7fed74198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed78972700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 0x7fed74198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44250/0 (socket says 172.21.15.110:44250) 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed78972700 1 -- 172.21.15.110:0/2467808224 learned_addr learned my addr 172.21.15.110:0/2467808224 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.966+0000 7fed79173700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed74103e30 0x7fed7419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed73fff700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed74103490 0x7fed74198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed78972700 1 -- 172.21.15.110:0/2467808224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed74103e30 msgr2=0x7fed7419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed78972700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed74103e30 0x7fed7419f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed78972700 1 -- 172.21.15.110:0/2467808224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed74103490 msgr2=0x7fed74198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed78972700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed74103490 0x7fed74198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed78972700 1 -- 172.21.15.110:0/2467808224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed741a0080 con 0x7fed74102290 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed78972700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 0x7fed74198420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fed68003680 tx=0x7fed680036b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed71ffb700 1 -- 172.21.15.110:0/2467808224 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68027070 con 0x7fed74102290 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed71ffb700 1 -- 172.21.15.110:0/2467808224 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed680041d0 con 0x7fed74102290 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed741a0310 con 0x7fed74102290 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed71ffb700 1 -- 172.21.15.110:0/2467808224 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6800f460 con 0x7fed74102290 2024-04-25T00:01:38.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.967+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed741a0870 con 0x7fed74102290 2024-04-25T00:01:38.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.968+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed74068f20 con 0x7fed74102290 2024-04-25T00:01:38.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.969+0000 7fed71ffb700 1 -- 172.21.15.110:0/2467808224 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fed6802a070 con 0x7fed74102290 2024-04-25T00:01:38.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.970+0000 7fed71ffb700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed640a2c20 0x7fed640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:38.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.970+0000 7fed73fff700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed640a2c20 0x7fed640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:38.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.970+0000 7fed71ffb700 1 -- 172.21.15.110:0/2467808224 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fed680d01b0 con 0x7fed74102290 2024-04-25T00:01:38.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.970+0000 7fed73fff700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed640a2c20 0x7fed640a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed60006fd0 tx=0x7fed60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:38.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:38.974+0000 7fed71ffb700 1 -- 172.21.15.110:0/2467808224 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fed6805e740 con 0x7fed74102290 2024-04-25T00:01:39.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.182+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fed74062670 con 0x7fed640a2c20 2024-04-25T00:01:39.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.196+0000 7fed71ffb700 1 -- 172.21.15.110:0/2467808224 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fed74062670 con 0x7fed640a2c20 2024-04-25T00:01:39.196 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T00:01:39.196 INFO:teuthology.orchestra.run.smithi110.stdout:mds.a smithi110 running (104s) 15s ago 104s 13.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef bf534df8e285 2024-04-25T00:01:39.196 INFO:teuthology.orchestra.run.smithi110.stdout:mds.b smithi173 running (101s) 35s ago 101s 26.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29446a726e8d 2024-04-25T00:01:39.196 INFO:teuthology.orchestra.run.smithi110.stdout:mds.c smithi188 running (116s) 35s ago 115s 20.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 05ee9d634df9 2024-04-25T00:01:39.196 INFO:teuthology.orchestra.run.smithi110.stdout:mds.d smithi110 running (2m) 15s ago 2m 13.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 9759b01d5621 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.e smithi173 running (2m) 35s ago 2m 16.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef a4b4b4bffc1b 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.f smithi188 running (118s) 35s ago 118s 19.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3ceaca1cfe29 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.g smithi110 running (107s) 15s ago 107s 16.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3178ef81328b 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.h smithi173 running (113s) 35s ago 112s 17.6M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1172453ed445 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.i smithi188 running (110s) 35s ago 110s 17.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef e29352a6f3e5 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.j smithi110 running (2m) 15s ago 2m 13.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1712dc0d25d9 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.k smithi173 running (2m) 35s ago 2m 13.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 54140fa47437 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mds.l smithi188 running (2m) 35s ago 2m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 5a226839493f 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.x smithi110 *:9283,8765 running (9m) 15s ago 9m 514M - 19.0.0-2445-gd2c45d97 f68e37b807ef 6592987f0c90 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.y smithi173 *:8443,8765 running (7m) 35s ago 7m 454M - 19.0.0-2445-gd2c45d97 f68e37b807ef ab3af38ff9f5 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.z smithi188 *:8443,8765 running (7m) 35s ago 7m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7ac6894c5fa2 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mon.a smithi110 running (9m) 15s ago 9m 51.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 0ca456db64e8 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mon.b smithi173 running (7m) 35s ago 7m 41.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef b9c86dc5242f 2024-04-25T00:01:39.197 INFO:teuthology.orchestra.run.smithi110.stdout:mon.c smithi188 running (7m) 35s ago 7m 42.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef c462243bdd47 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (7m) 15s ago 7m 44.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 27937edf1e3a 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi173 running (6m) 35s ago 6m 45.6M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef f823e647956b 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi188 running (6m) 35s ago 6m 57.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 53baacc260e0 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi110 running (6m) 15s ago 5m 54.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 54ba0f9ea16a 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi173 running (5m) 35s ago 5m 42.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef b505068aca92 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi188 running (5m) 35s ago 5m 43.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef c0255818ac96 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.6 smithi110 running (4m) 15s ago 4m 42.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 3f772af33bfb 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.7 smithi173 running (4m) 35s ago 4m 43.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef f3c595acd8c8 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.8 smithi188 running (3m) 35s ago 3m 52.6M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef d03a6a594ba5 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.9 smithi110 running (3m) 15s ago 3m 44.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef e20472883720 2024-04-25T00:01:39.198 INFO:teuthology.orchestra.run.smithi110.stdout:osd.10 smithi173 running (2m) 35s ago 2m 44.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 5a4b8ebf3023 2024-04-25T00:01:39.199 INFO:teuthology.orchestra.run.smithi110.stdout:osd.11 smithi188 running (2m) 35s ago 2m 42.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 4532eb270b10 2024-04-25T00:01:39.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed640a2c20 msgr2=0x7fed640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed640a2c20 0x7fed640a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed60006fd0 tx=0x7fed60008040 comp rx=0 tx=0).stop 2024-04-25T00:01:39.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 msgr2=0x7fed74198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:39.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 0x7fed74198420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fed68003680 tx=0x7fed680036b0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 shutdown_connections 2024-04-25T00:01:39.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed640a2c20 0x7fed640a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed74103e30 0x7fed7419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed74102290 0x7fed74198420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 --2- 172.21.15.110:0/2467808224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed74103490 0x7fed74198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:39.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.198+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 >> 172.21.15.110:0/2467808224 conn(0x7fed740fd830 msgr2=0x7fed741067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:39.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.199+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 shutdown_connections 2024-04-25T00:01:39.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:39.199+0000 7fed7abd6700 1 -- 172.21.15.110:0/2467808224 wait complete. 2024-04-25T00:01:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:39 smithi188 ceph-mon[33285]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:01:40.022 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:39 smithi110 ceph-mon[29187]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:01:40.054 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T00:01:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:39 smithi173 ceph-mon[32402]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:01:40.293 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:41.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:40 smithi188 ceph-mon[33285]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:41.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:40 smithi110 ceph-mon[29187]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:40 smithi173 ceph-mon[32402]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:41.786 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:41 smithi110 ceph-mon[29187]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:41 smithi188 ceph-mon[33285]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:42.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 -- 172.21.15.110:0/627353176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103490 msgr2=0x7f02e01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/627353176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103490 0x7f02e01038f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0099c0 tx=0x7f02cc009cd0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 -- 172.21.15.110:0/627353176 shutdown_connections 2024-04-25T00:01:42.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/627353176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02e0103e30 0x7f02e010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/627353176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02e0102290 0x7f02e01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/627353176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103490 0x7f02e01038f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 -- 172.21.15.110:0/627353176 >> 172.21.15.110:0/627353176 conn(0x7f02e00fd830 msgr2=0x7f02e00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 -- 172.21.15.110:0/627353176 shutdown_connections 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.042+0000 7f02e6bb5700 1 -- 172.21.15.110:0/627353176 wait complete. 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e6bb5700 1 Processor -- start 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e6bb5700 1 -- start start 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e6bb5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02e0102290 0x7f02e0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e6bb5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02e0103490 0x7f02e0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e6bb5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103e30 0x7f02e019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e6bb5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02e010dd80 con 0x7f02e0103e30 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e6bb5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f02e010dc00 con 0x7f02e0103490 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e6bb5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f02e010df00 con 0x7f02e0102290 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e5152700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103e30 0x7f02e019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02dffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02e0103490 0x7f02e0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e5152700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103e30 0x7f02e019f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44276/0 (socket says 172.21.15.110:44276) 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e5152700 1 -- 172.21.15.110:0/2368852440 learned_addr learned my addr 172.21.15.110:0/2368852440 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e4951700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02e0102290 0x7f02e0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e5152700 1 -- 172.21.15.110:0/2368852440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02e0102290 msgr2=0x7f02e0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e5152700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02e0102290 0x7f02e0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e5152700 1 -- 172.21.15.110:0/2368852440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02e0103490 msgr2=0x7f02e0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e5152700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02e0103490 0x7f02e0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e5152700 1 -- 172.21.15.110:0/2368852440 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e01a0060 con 0x7f02e0103e30 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.043+0000 7f02e4951700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02e0102290 0x7f02e0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.044+0000 7f02e5152700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103e30 0x7f02e019f9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f02d800b9a0 tx=0x7f02d800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.044+0000 7f02ddffb700 1 -- 172.21.15.110:0/2368852440 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8019070 con 0x7f02e0103e30 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.044+0000 7f02ddffb700 1 -- 172.21.15.110:0/2368852440 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02d800cde0 con 0x7f02e0103e30 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.044+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e01a0350 con 0x7f02e0103e30 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.044+0000 7f02ddffb700 1 -- 172.21.15.110:0/2368852440 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8007370 con 0x7f02e0103e30 2024-04-25T00:01:42.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.044+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e01a0960 con 0x7f02e0103e30 2024-04-25T00:01:42.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.045+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e0068f20 con 0x7f02e0103e30 2024-04-25T00:01:42.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.046+0000 7f02ddffb700 1 -- 172.21.15.110:0/2368852440 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f02d801c070 con 0x7f02e0103e30 2024-04-25T00:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.046+0000 7f02ddffb700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02d00a2ca0 0x7f02d00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.046+0000 7f02e4951700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02d00a2ca0 0x7f02d00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.046+0000 7f02ddffb700 1 -- 172.21.15.110:0/2368852440 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f02d80ce530 con 0x7f02e0103e30 2024-04-25T00:01:42.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.047+0000 7f02e4951700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02d00a2ca0 0x7f02d00a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:42.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.051+0000 7f02ddffb700 1 -- 172.21.15.110:0/2368852440 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f02d809b4c0 con 0x7f02e0103e30 2024-04-25T00:01:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:41 smithi173 ceph-mon[32402]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:42.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.255+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f02e0062670 con 0x7f02d00a2ca0 2024-04-25T00:01:42.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.262+0000 7f02ddffb700 1 -- 172.21.15.110:0/2368852440 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f02e0062670 con 0x7f02d00a2ca0 2024-04-25T00:01:42.262 INFO:teuthology.orchestra.run.smithi110.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T00:01:42.262 INFO:teuthology.orchestra.run.smithi110.stdout:mds.all 12/12 38s ago 2m smithi110=a;smithi110=d;smithi110=g;smithi110=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-04-25T00:01:42.262 INFO:teuthology.orchestra.run.smithi110.stdout:mgr 3/3 38s ago 7m smithi110=x;smithi173=y;smithi188=z;count:3 2024-04-25T00:01:42.262 INFO:teuthology.orchestra.run.smithi110.stdout:mon 3/3 38s ago 8m smithi110:172.21.15.110=a;smithi173:172.21.15.173=b;smithi188:172.21.15.188=c;count:3 2024-04-25T00:01:42.262 INFO:teuthology.orchestra.run.smithi110.stdout:osd 12 38s ago - 2024-04-25T00:01:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02d00a2ca0 msgr2=0x7f02d00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02d00a2ca0 0x7f02d00a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d4005dc0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103e30 msgr2=0x7f02e019f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103e30 0x7f02e019f9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f02d800b9a0 tx=0x7f02d800bcb0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 shutdown_connections 2024-04-25T00:01:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02d00a2ca0 0x7f02d00a5160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02e0103e30 0x7f02e019f9c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02e0102290 0x7f02e0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 --2- 172.21.15.110:0/2368852440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02e0103490 0x7f02e0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 >> 172.21.15.110:0/2368852440 conn(0x7f02e00fd830 msgr2=0x7f02e01068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 shutdown_connections 2024-04-25T00:01:42.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:42.264+0000 7f02e6bb5700 1 -- 172.21.15.110:0/2368852440 wait complete. 2024-04-25T00:01:42.986 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T00:01:43.226 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:43 smithi188 ceph-mon[33285]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:43 smithi188 ceph-mon[33285]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:43 smithi110 ceph-mon[29187]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:43 smithi110 ceph-mon[29187]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:43 smithi173 ceph-mon[32402]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:43 smithi173 ceph-mon[32402]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.150+0000 7fd9ba129700 1 -- 172.21.15.110:0/2819158244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 msgr2=0x7fd9b4102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.150+0000 7fd9ba129700 1 --2- 172.21.15.110:0/2819158244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 0x7fd9b4102be0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009990 tx=0x7fd9a4009ca0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.150+0000 7fd9ba129700 1 -- 172.21.15.110:0/2819158244 shutdown_connections 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.150+0000 7fd9ba129700 1 --2- 172.21.15.110:0/2819158244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9b4101e10 0x7fd9b41096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.150+0000 7fd9ba129700 1 --2- 172.21.15.110:0/2819158244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 0x7fd9b4102be0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.150+0000 7fd9ba129700 1 --2- 172.21.15.110:0/2819158244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b4101470 0x7fd9b41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.151+0000 7fd9ba129700 1 -- 172.21.15.110:0/2819158244 >> 172.21.15.110:0/2819158244 conn(0x7fd9b40fbf00 msgr2=0x7fd9b40fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.151+0000 7fd9ba129700 1 -- 172.21.15.110:0/2819158244 shutdown_connections 2024-04-25T00:01:45.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.151+0000 7fd9ba129700 1 -- 172.21.15.110:0/2819158244 wait complete. 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.151+0000 7fd9ba129700 1 Processor -- start 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.151+0000 7fd9ba129700 1 -- start start 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9ba129700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b4101470 0x7fd9b41a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9ba129700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9b4101e10 0x7fd9b41a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9ba129700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 0x7fd9b41a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9ba129700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410ccd0 con 0x7fd9b41027c0 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9ba129700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410cb50 con 0x7fd9b4101e10 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9ba129700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410ce50 con 0x7fd9b4101470 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 0x7fd9b41a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 0x7fd9b41a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44304/0 (socket says 172.21.15.110:44304) 2024-04-25T00:01:45.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 -- 172.21.15.110:0/4219258951 learned_addr learned my addr 172.21.15.110:0/4219258951 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b37fe700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b4101470 0x7fd9b41a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b2ffd700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9b4101e10 0x7fd9b41a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 -- 172.21.15.110:0/4219258951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b4101470 msgr2=0x7fd9b41a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b4101470 0x7fd9b41a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 -- 172.21.15.110:0/4219258951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9b4101e10 msgr2=0x7fd9b41a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9b4101e10 0x7fd9b41a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 -- 172.21.15.110:0/4219258951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b41a6660 con 0x7fd9b41027c0 2024-04-25T00:01:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b3fff700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 0x7fd9b41a5d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b970 tx=0x7fd9ac00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.152+0000 7fd9b0ff9700 1 -- 172.21.15.110:0/4219258951 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac019070 con 0x7fd9b41027c0 2024-04-25T00:01:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.153+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b41a6950 con 0x7fd9b41027c0 2024-04-25T00:01:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.153+0000 7fd9b0ff9700 1 -- 172.21.15.110:0/4219258951 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9ac014440 con 0x7fd9b41027c0 2024-04-25T00:01:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.153+0000 7fd9b0ff9700 1 -- 172.21.15.110:0/4219258951 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00ce60 con 0x7fd9b41027c0 2024-04-25T00:01:45.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.153+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41a6e90 con 0x7fd9b41027c0 2024-04-25T00:01:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.154+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b4068f20 con 0x7fd9b41027c0 2024-04-25T00:01:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.155+0000 7fd9b0ff9700 1 -- 172.21.15.110:0/4219258951 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd9ac01c070 con 0x7fd9b41027c0 2024-04-25T00:01:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.156+0000 7fd9b0ff9700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd99c0a2c20 0x7fd99c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.156+0000 7fd9b37fe700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd99c0a2c20 0x7fd99c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.156+0000 7fd9b0ff9700 1 -- 172.21.15.110:0/4219258951 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd9ac0ce960 con 0x7fd9b41027c0 2024-04-25T00:01:45.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.157+0000 7fd9b37fe700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd99c0a2c20 0x7fd99c0a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4109be0 tx=0x7fd9a4003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:45.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.160+0000 7fd9b0ff9700 1 -- 172.21.15.110:0/4219258951 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd9ac09b8f0 con 0x7fd9b41027c0 2024-04-25T00:01:45.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.360+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fd9b4100a70 con 0x7fd99c0a2c20 2024-04-25T00:01:45.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.362+0000 7fd9b0ff9700 1 -- 172.21.15.110:0/4219258951 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fd9b4100a70 con 0x7fd99c0a2c20 2024-04-25T00:01:45.362 INFO:teuthology.orchestra.run.smithi110.stdout:HOST ADDR LABELS STATUS 2024-04-25T00:01:45.362 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 172.21.15.110 2024-04-25T00:01:45.362 INFO:teuthology.orchestra.run.smithi110.stdout:smithi173 172.21.15.173 2024-04-25T00:01:45.362 INFO:teuthology.orchestra.run.smithi110.stdout:smithi188 172.21.15.188 2024-04-25T00:01:45.363 INFO:teuthology.orchestra.run.smithi110.stdout:3 hosts in cluster 2024-04-25T00:01:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd99c0a2c20 msgr2=0x7fd99c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd99c0a2c20 0x7fd99c0a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4109be0 tx=0x7fd9a4003680 comp rx=0 tx=0).stop 2024-04-25T00:01:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 msgr2=0x7fd9b41a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 0x7fd9b41a5d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b970 tx=0x7fd9ac00bd30 comp rx=0 tx=0).stop 2024-04-25T00:01:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 shutdown_connections 2024-04-25T00:01:45.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd99c0a2c20 0x7fd99c0a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9b41027c0 0x7fd9b41a5d90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b4101470 0x7fd9b41a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 --2- 172.21.15.110:0/4219258951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9b4101e10 0x7fd9b41a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:45.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 >> 172.21.15.110:0/4219258951 conn(0x7fd9b40fbf00 msgr2=0x7fd9b4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:45.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 shutdown_connections 2024-04-25T00:01:45.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:45.364+0000 7fd9ba129700 1 -- 172.21.15.110:0/4219258951 wait complete. 2024-04-25T00:01:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:45 smithi188 ceph-mon[33285]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:45 smithi110 ceph-mon[29187]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:45 smithi173 ceph-mon[32402]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:46.257 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T00:01:46.502 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:46.895 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:46 smithi110 ceph-mon[29187]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:47.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:46 smithi188 ceph-mon[33285]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:47.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:46 smithi173 ceph-mon[32402]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:47.952 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:47 smithi110 ceph-mon[29187]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:47 smithi188 ceph-mon[33285]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:47 smithi173 ceph-mon[32402]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:48.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 -- 172.21.15.110:0/294930100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa270102c60 msgr2=0x7fa270109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 --2- 172.21.15.110:0/294930100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa270102c60 0x7fa270109510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b0d0 tx=0x7fa26800b3e0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 -- 172.21.15.110:0/294930100 shutdown_connections 2024-04-25T00:01:48.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 --2- 172.21.15.110:0/294930100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa270102c60 0x7fa270109510 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 --2- 172.21.15.110:0/294930100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2701010c0 0x7fa2701014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 --2- 172.21.15.110:0/294930100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2701022c0 0x7fa270102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 -- 172.21.15.110:0/294930100 >> 172.21.15.110:0/294930100 conn(0x7fa2700fc640 msgr2=0x7fa2700feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 -- 172.21.15.110:0/294930100 shutdown_connections 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.335+0000 7fa2757a0700 1 -- 172.21.15.110:0/294930100 wait complete. 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa2757a0700 1 Processor -- start 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa2757a0700 1 -- start start 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa2757a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2701010c0 0x7fa27006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa2757a0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2701022c0 0x7fa27019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa2757a0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa270102c60 0x7fa27019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa2757a0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa27010cb60 con 0x7fa2701010c0 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa2757a0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa27010c9e0 con 0x7fa270102c60 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa2757a0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa27010cce0 con 0x7fa2701022c0 2024-04-25T00:01:48.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa26ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2701010c0 0x7fa27006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa26ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2701010c0 0x7fa27006aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40796/0 (socket says 172.21.15.110:40796) 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa26ffff700 1 -- 172.21.15.110:0/1459129558 learned_addr learned my addr 172.21.15.110:0/1459129558 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa274f9f700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa270102c60 0x7fa27019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa26f7fe700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2701022c0 0x7fa27019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.336+0000 7fa26ffff700 1 -- 172.21.15.110:0/1459129558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2701022c0 msgr2=0x7fa27019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa26ffff700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2701022c0 0x7fa27019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa26ffff700 1 -- 172.21.15.110:0/1459129558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa270102c60 msgr2=0x7fa27019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa26ffff700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa270102c60 0x7fa27019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa26ffff700 1 -- 172.21.15.110:0/1459129558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27019fe50 con 0x7fa2701010c0 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa274f9f700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa270102c60 0x7fa27019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa26ffff700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2701010c0 0x7fa27006aa70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa25800d980 tx=0x7fa25800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:48.341 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa26d7fa700 1 -- 172.21.15.110:0/1459129558 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258017070 con 0x7fa2701010c0 2024-04-25T00:01:48.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2701a0140 con 0x7fa2701010c0 2024-04-25T00:01:48.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa26d7fa700 1 -- 172.21.15.110:0/1459129558 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa258010460 con 0x7fa2701010c0 2024-04-25T00:01:48.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa26d7fa700 1 -- 172.21.15.110:0/1459129558 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25800de60 con 0x7fa2701010c0 2024-04-25T00:01:48.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.337+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2701a0680 con 0x7fa2701010c0 2024-04-25T00:01:48.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.338+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa270068f20 con 0x7fa2701010c0 2024-04-25T00:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.340+0000 7fa26d7fa700 1 -- 172.21.15.110:0/1459129558 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa258010610 con 0x7fa2701010c0 2024-04-25T00:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.340+0000 7fa26d7fa700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.341+0000 7fa26f7fe700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.341+0000 7fa26d7fa700 1 -- 172.21.15.110:0/1459129558 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa2580cebe0 con 0x7fa2701010c0 2024-04-25T00:01:48.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.341+0000 7fa26f7fe700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa27006aff0 tx=0x7fa260005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:48.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.344+0000 7fa26d7fa700 1 -- 172.21.15.110:0/1459129558 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa25809bb10 con 0x7fa2701010c0 2024-04-25T00:01:48.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.560+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 --> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa270062670 con 0x7fa25c0a2c20 2024-04-25T00:01:48.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.562+0000 7fa26d7fa700 1 -- 172.21.15.110:0/1459129558 <== mgr.14152 v2:172.21.15.110:6800/546485163 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa270062670 con 0x7fa25c0a2c20 2024-04-25T00:01:48.562 INFO:teuthology.orchestra.run.smithi110.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T00:01:48.563 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:01:48.563 INFO:teuthology.orchestra.run.smithi110.stdout:smithi173 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204019S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:01:48.563 INFO:teuthology.orchestra.run.smithi110.stdout:smithi188 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MP400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.564+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa25c0a2c20 msgr2=0x7fa25c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.564+0000 7fa2757a0700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa27006aff0 tx=0x7fa260005cf0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.564+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2701010c0 msgr2=0x7fa27006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.564+0000 7fa2757a0700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2701010c0 0x7fa27006aa70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa25800d980 tx=0x7fa25800dd40 comp rx=0 tx=0).stop 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.565+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 shutdown_connections 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.565+0000 7fa2757a0700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa25c0a2c20 0x7fa25c0a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.565+0000 7fa2757a0700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa270102c60 0x7fa27019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.565+0000 7fa2757a0700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2701010c0 0x7fa27006aa70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.565+0000 7fa2757a0700 1 --2- 172.21.15.110:0/1459129558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2701022c0 0x7fa27019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.565+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 >> 172.21.15.110:0/1459129558 conn(0x7fa2700fc640 msgr2=0x7fa270105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.565+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 shutdown_connections 2024-04-25T00:01:48.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:48.565+0000 7fa2757a0700 1 -- 172.21.15.110:0/1459129558 wait complete. 2024-04-25T00:01:49.422 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:01:49.440 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T00:01:49.441 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T00:01:49.681 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:49.950 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:49 smithi110 ceph-mon[29187]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:49.950 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:49 smithi110 ceph-mon[29187]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:50.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:49 smithi188 ceph-mon[33285]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:50.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:49 smithi188 ceph-mon[33285]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:49 smithi173 ceph-mon[32402]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:49 smithi173 ceph-mon[32402]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:01:51.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 -- 172.21.15.110:0/924176594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0102d10 msgr2=0x7effe0105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 --2- 172.21.15.110:0/924176594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0102d10 0x7effe0105100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7effd0009990 tx=0x7effd0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 -- 172.21.15.110:0/924176594 shutdown_connections 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 --2- 172.21.15.110:0/924176594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe0107f70 0x7effe010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 --2- 172.21.15.110:0/924176594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0102d10 0x7effe0105100 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 --2- 172.21.15.110:0/924176594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe0105640 0x7effe0107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 -- 172.21.15.110:0/924176594 >> 172.21.15.110:0/924176594 conn(0x7effe00fc700 msgr2=0x7effe00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 -- 172.21.15.110:0/924176594 shutdown_connections 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 -- 172.21.15.110:0/924176594 wait complete. 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.484+0000 7effe486e700 1 Processor -- start 2024-04-25T00:01:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effe486e700 1 -- start start 2024-04-25T00:01:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effe486e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe0102d10 0x7effe0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effe486e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe0105640 0x7effe0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effe486e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0107f70 0x7effe019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effe486e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7effe010dd80 con 0x7effe0107f70 2024-04-25T00:01:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effe486e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7effe010dc00 con 0x7effe0105640 2024-04-25T00:01:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effe486e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7effe010df00 con 0x7effe0102d10 2024-04-25T00:01:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0107f70 0x7effe019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0107f70 0x7effe019f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40806/0 (socket says 172.21.15.110:40806) 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 -- 172.21.15.110:0/1161458977 learned_addr learned my addr 172.21.15.110:0/1161458977 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effde59c700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe0102d10 0x7effe0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effd5d9b700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe0105640 0x7effe0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 -- 172.21.15.110:0/1161458977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe0102d10 msgr2=0x7effe0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe0102d10 0x7effe0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 -- 172.21.15.110:0/1161458977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe0105640 msgr2=0x7effe0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe0105640 0x7effe0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 -- 172.21.15.110:0/1161458977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe019ffa0 con 0x7effe0107f70 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.485+0000 7effded9d700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0107f70 0x7effe019f900 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7effd800ba10 tx=0x7effd800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.486+0000 7effd7fff700 1 -- 172.21.15.110:0/1161458977 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8019070 con 0x7effe0107f70 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.486+0000 7effd7fff700 1 -- 172.21.15.110:0/1161458977 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effd8014440 con 0x7effe0107f70 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.486+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe01a0290 con 0x7effe0107f70 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.486+0000 7effd7fff700 1 -- 172.21.15.110:0/1161458977 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd800ce60 con 0x7effe0107f70 2024-04-25T00:01:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.486+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe01a0820 con 0x7effe0107f70 2024-04-25T00:01:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.487+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe0068f20 con 0x7effe0107f70 2024-04-25T00:01:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.488+0000 7effd7fff700 1 -- 172.21.15.110:0/1161458977 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7effd801c070 con 0x7effe0107f70 2024-04-25T00:01:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.488+0000 7effd7fff700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7effcc0a2ca0 0x7effcc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.488+0000 7effde59c700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7effcc0a2ca0 0x7effcc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.488+0000 7effd7fff700 1 -- 172.21.15.110:0/1161458977 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7effd80ce330 con 0x7effe0107f70 2024-04-25T00:01:51.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.489+0000 7effde59c700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7effcc0a2ca0 0x7effcc0a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7effd000b5c0 tx=0x7effd0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:51.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.493+0000 7effd7fff700 1 -- 172.21.15.110:0/1161458977 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7effd809b350 con 0x7effe0107f70 2024-04-25T00:01:51.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.733+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7effe0062e40 con 0x7effe0107f70 2024-04-25T00:01:51.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.734+0000 7effd7fff700 1 -- 172.21.15.110:0/1161458977 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3058 (secure 0 0 0) 0x7effd809ac50 con 0x7effe0107f70 2024-04-25T00:01:51.734 INFO:teuthology.orchestra.run.smithi110.stdout:e21 2024-04-25T00:01:51.734 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:01:51.734 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 21 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T00:00:02.856564+0000 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T00:00:07.553801+0000 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 300 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T00:01:51.735 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 3 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:in 0,1,2 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=24512,1=24476,2=24457} 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 0 2024-04-25T00:01:51.736 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 24512 members: 24457,24476,24512 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.b{0:24512} state up:active seq 3 addr [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.c{0:24442} state up:standby-replay seq 1 addr [v2:172.21.15.188:6836/1170231083,v1:172.21.15.188:6837/1170231083] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.h{1:24476} state up:active seq 6 addr [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.e{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.173:6834/1627462282,v1:172.21.15.173:6835/1627462282] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.i{2:24457} state up:active seq 6 addr [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.f{2:24427} state up:standby-replay seq 1 addr [v2:172.21.15.188:6834/946560580,v1:172.21.15.188:6835/946560580] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.110:6834/1261890335,v1:172.21.15.110:6835/1261890335] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.110:6836/1723817194,v1:172.21.15.110:6837/1723817194] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.g{-1:14619} state up:standby seq 1 addr [v2:172.21.15.110:6838/717154296,v1:172.21.15.110:6839/717154296] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.a{-1:14637} state up:standby seq 1 addr [v2:172.21.15.110:6840/2788119142,v1:172.21.15.110:6841/2788119142] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.737 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.l{-1:24391} state up:standby seq 1 addr [v2:172.21.15.188:6832/2454659277,v1:172.21.15.188:6833/2454659277] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.738 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.k{-1:24419} state up:standby seq 1 addr [v2:172.21.15.173:6832/373959969,v1:172.21.15.173:6833/373959969] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:01:51.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7effcc0a2ca0 msgr2=0x7effcc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7effcc0a2ca0 0x7effcc0a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7effd000b5c0 tx=0x7effd0003680 comp rx=0 tx=0).stop 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0107f70 msgr2=0x7effe019f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0107f70 0x7effe019f900 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7effd800ba10 tx=0x7effd800bdd0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 shutdown_connections 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7effcc0a2ca0 0x7effcc0a5160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effe0107f70 0x7effe019f900 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effe0102d10 0x7effe0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 --2- 172.21.15.110:0/1161458977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe0105640 0x7effe0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 >> 172.21.15.110:0/1161458977 conn(0x7effe00fc700 msgr2=0x7effe0103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 shutdown_connections 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:51.737+0000 7effe486e700 1 -- 172.21.15.110:0/1161458977 wait complete. 2024-04-25T00:01:51.739 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-04-25T00:01:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:51 smithi188 ceph-mon[33285]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:51 smithi110 ceph-mon[29187]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:51 smithi173 ceph-mon[32402]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:52.585 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T00:01:52.828 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:01:52.951 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:52 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1161458977' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:01:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:52 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1161458977' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:01:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:52 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1161458977' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:01:53.990 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:53 smithi110 ceph-mon[29187]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:53 smithi188 ceph-mon[33285]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:53 smithi173 ceph-mon[32402]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.612+0000 7f12703b3700 1 -- 172.21.15.110:0/1177255470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268101190 msgr2=0x7f12681015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.612+0000 7f12703b3700 1 --2- 172.21.15.110:0/1177255470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268101190 0x7f12681015b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12640099c0 tx=0x7f1264009cd0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.612+0000 7f12703b3700 1 -- 172.21.15.110:0/1177255470 shutdown_connections 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.612+0000 7f12703b3700 1 --2- 172.21.15.110:0/1177255470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1268102d30 0x7f12681095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.612+0000 7f12703b3700 1 --2- 172.21.15.110:0/1177255470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268101190 0x7f12681015b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.612+0000 7f12703b3700 1 --2- 172.21.15.110:0/1177255470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268102390 0x7f12681027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.612+0000 7f12703b3700 1 -- 172.21.15.110:0/1177255470 >> 172.21.15.110:0/1177255470 conn(0x7f12680fc730 msgr2=0x7f12680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 -- 172.21.15.110:0/1177255470 shutdown_connections 2024-04-25T00:01:54.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 -- 172.21.15.110:0/1177255470 wait complete. 2024-04-25T00:01:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 Processor -- start 2024-04-25T00:01:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 -- start start 2024-04-25T00:01:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268101190 0x7f1268198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1268102390 0x7f1268198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268102d30 0x7f126819fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f126810cc40 con 0x7f1268102d30 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f126810cac0 con 0x7f1268101190 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f12703b3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f126810cdc0 con 0x7f1268102390 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.613+0000 7f126e950700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268102d30 0x7f126819fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e950700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268102d30 0x7f126819fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40832/0 (socket says 172.21.15.110:40832) 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e950700 1 -- 172.21.15.110:0/3014115132 learned_addr learned my addr 172.21.15.110:0/3014115132 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e14f700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268101190 0x7f1268198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126d94e700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1268102390 0x7f1268198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e950700 1 -- 172.21.15.110:0/3014115132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1268102390 msgr2=0x7f1268198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e950700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1268102390 0x7f1268198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e950700 1 -- 172.21.15.110:0/3014115132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268101190 msgr2=0x7f1268198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e950700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268101190 0x7f1268198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e950700 1 -- 172.21.15.110:0/3014115132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12681a00a0 con 0x7f1268102d30 2024-04-25T00:01:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126d94e700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1268102390 0x7f1268198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f126e950700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268102d30 0x7f126819fa00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1260007f10 tx=0x7f126000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f125f7fe700 1 -- 172.21.15.110:0/3014115132 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260004bb0 con 0x7f1268102d30 2024-04-25T00:01:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12681a0390 con 0x7f1268102d30 2024-04-25T00:01:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f125f7fe700 1 -- 172.21.15.110:0/3014115132 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1260004d50 con 0x7f1268102d30 2024-04-25T00:01:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f125f7fe700 1 -- 172.21.15.110:0/3014115132 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260005990 con 0x7f1268102d30 2024-04-25T00:01:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.614+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681a0880 con 0x7f1268102d30 2024-04-25T00:01:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.615+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1268068f20 con 0x7f1268102d30 2024-04-25T00:01:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.616+0000 7f125f7fe700 1 -- 172.21.15.110:0/3014115132 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f126001a020 con 0x7f1268102d30 2024-04-25T00:01:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.617+0000 7f125f7fe700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12540a2c20 0x7f12540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.617+0000 7f126e14f700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12540a2c20 0x7f12540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.617+0000 7f125f7fe700 1 -- 172.21.15.110:0/3014115132 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f12600cee50 con 0x7f1268102d30 2024-04-25T00:01:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.617+0000 7f126e14f700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12540a2c20 0x7f12540a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1264000c00 tx=0x7f1264005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:54.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.620+0000 7f125f7fe700 1 -- 172.21.15.110:0/3014115132 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1260010070 con 0x7f1268102d30 2024-04-25T00:01:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.820+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f12681a0ca0 con 0x7f1268102d30 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.821+0000 7f125f7fe700 1 -- 172.21.15.110:0/3014115132 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5281 (secure 0 0 0) 0x7f126009bde0 con 0x7f1268102d30 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 87 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:fsid 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-24T23:51:53.430143+0000 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T00:00:55.648110+0000 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:crush_version 26 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:full_ratio 0.95 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:backfillfull_ratio 0.9 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:nearfull_ratio 0.85 2024-04-25T00:01:54.821 INFO:teuthology.orchestra.run.smithi110.stdout:require_min_compat_client luminous 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:min_compat_client luminous 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:require_osd_release squid 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:stretch_mode_enabled false 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:max_osd 12 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6802/1941809226,v1:172.21.15.110:6803/1941809226] [v2:172.21.15.110:6804/1941809226,v1:172.21.15.110:6805/1941809226] exists,up e6eba556-dc44-4043-a0d7-c9d6dc839473 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/1633800415,v1:172.21.15.173:6801/1633800415] [v2:172.21.15.173:6802/1633800415,v1:172.21.15.173:6803/1633800415] exists,up fbfc4554-e9a2-4cc3-a392-cfe42b675b63 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/2140926702,v1:172.21.15.188:6801/2140926702] [v2:172.21.15.188:6802/2140926702,v1:172.21.15.188:6803/2140926702] exists,up 3c3e4437-7088-4060-a771-70bebf386247 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6810/2506639922,v1:172.21.15.110:6811/2506639922] [v2:172.21.15.110:6812/2506639922,v1:172.21.15.110:6813/2506639922] exists,up 307060eb-b8bb-4b80-b43e-cc22c933400f 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/3034286284,v1:172.21.15.173:6809/3034286284] [v2:172.21.15.173:6810/3034286284,v1:172.21.15.173:6811/3034286284] exists,up 12c128f9-0d7a-44ff-96c1-b2e8417851d3 2024-04-25T00:01:54.822 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/216258077,v1:172.21.15.188:6809/216258077] [v2:172.21.15.188:6810/216258077,v1:172.21.15.188:6811/216258077] exists,up 6b479fe2-2120-4b2c-a0d6-986d5d7582d0 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6818/3643107507,v1:172.21.15.110:6819/3643107507] [v2:172.21.15.110:6820/3643107507,v1:172.21.15.110:6821/3643107507] exists,up 9bf7fd3c-3d98-411e-b50b-bbe5ea52f664 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/3085868439,v1:172.21.15.173:6817/3085868439] [v2:172.21.15.173:6818/3085868439,v1:172.21.15.173:6819/3085868439] exists,up 19d1620c-d00e-4c4a-b0a3-acf84b3021ec 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/879002066,v1:172.21.15.188:6817/879002066] [v2:172.21.15.188:6818/879002066,v1:172.21.15.188:6819/879002066] exists,up b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6826/453265858,v1:172.21.15.110:6827/453265858] [v2:172.21.15.110:6828/453265858,v1:172.21.15.110:6829/453265858] exists,up 1d5cf0eb-8bf8-488a-8183-f8c9961006f7 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/3803887034,v1:172.21.15.173:6825/3803887034] [v2:172.21.15.173:6826/3803887034,v1:172.21.15.173:6827/3803887034] exists,up 6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/785596824,v1:172.21.15.188:6825/785596824] [v2:172.21.15.188:6826/785596824,v1:172.21.15.188:6827/785596824] exists,up 1f2ae40f-4939-417d-b66f-990914900d30 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:pg_upmap_items 2.0 [8,4] 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:pg_upmap_items 2.e [2,4] 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:pg_upmap_items 2.1c [8,11] 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:pg_upmap_items 3.13 [4,9] 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:pg_upmap_items 3.1e [3,9] 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:pg_upmap_items 3.3c [4,9] 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:0/2804791885 expires 2024-04-25T23:52:54.426106+0000 2024-04-25T00:01:54.823 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:6801/1818468115 expires 2024-04-25T23:52:54.426106+0000 2024-04-25T00:01:54.824 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:0/3421586240 expires 2024-04-25T23:52:54.426106+0000 2024-04-25T00:01:54.824 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:0/3060209766 expires 2024-04-25T23:52:27.865188+0000 2024-04-25T00:01:54.824 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:6800/2433943234 expires 2024-04-25T23:52:27.865188+0000 2024-04-25T00:01:54.824 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:6801/2433943234 expires 2024-04-25T23:52:27.865188+0000 2024-04-25T00:01:54.824 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:0/2783100792 expires 2024-04-25T23:52:27.865188+0000 2024-04-25T00:01:54.824 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:0/1429818736 expires 2024-04-25T23:52:54.426106+0000 2024-04-25T00:01:54.824 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:0/1657096513 expires 2024-04-25T23:52:27.865188+0000 2024-04-25T00:01:54.824 INFO:teuthology.orchestra.run.smithi110.stdout:blocklist 172.21.15.110:6800/1818468115 expires 2024-04-25T23:52:54.426106+0000 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12540a2c20 msgr2=0x7f12540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12540a2c20 0x7f12540a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1264000c00 tx=0x7f1264005fb0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268102d30 msgr2=0x7f126819fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268102d30 0x7f126819fa00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1260007f10 tx=0x7f126000d3b0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 shutdown_connections 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f12540a2c20 0x7f12540a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268102d30 0x7f126819fa00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268101190 0x7f1268198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 --2- 172.21.15.110:0/3014115132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1268102390 0x7f1268198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:54.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 >> 172.21.15.110:0/3014115132 conn(0x7f12680fc730 msgr2=0x7f12680fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:54.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 shutdown_connections 2024-04-25T00:01:54.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:54.823+0000 7f12703b3700 1 -- 172.21.15.110:0/3014115132 wait complete. 2024-04-25T00:01:55.647 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T00:01:55.706 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:55.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 -- 172.21.15.110:0/380247262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 msgr2=0x7f6808100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 --2- 172.21.15.110:0/380247262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 0x7f6808100ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009900 tx=0x7f67fc02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 -- 172.21.15.110:0/380247262 shutdown_connections 2024-04-25T00:01:55.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 --2- 172.21.15.110:0/380247262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6808102250 0x7f6808108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 --2- 172.21.15.110:0/380247262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6808101890 0x7f6808101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 --2- 172.21.15.110:0/380247262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 0x7f6808100ac0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 -- 172.21.15.110:0/380247262 >> 172.21.15.110:0/380247262 conn(0x7f68080fbc30 msgr2=0x7f68080fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:55.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 -- 172.21.15.110:0/380247262 shutdown_connections 2024-04-25T00:01:55.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 -- 172.21.15.110:0/380247262 wait complete. 2024-04-25T00:01:55.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.811+0000 7f680ed63640 1 Processor -- start 2024-04-25T00:01:55.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680ed63640 1 -- start start 2024-04-25T00:01:55.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680ed63640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 0x7f680806a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680ed63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6808101890 0x7f680819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680ed63640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6808102250 0x7f680819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680ed63640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f680810c1a0 con 0x7f68081006a0 2024-04-25T00:01:55.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680ed63640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f680810c020 con 0x7f6808102250 2024-04-25T00:01:55.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680ed63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f680810c320 con 0x7f6808101890 2024-04-25T00:01:55.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 0x7f680806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 0x7f680806a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40860/0 (socket says 172.21.15.110:40860) 2024-04-25T00:01:55.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 -- 172.21.15.110:0/2350249123 learned_addr learned my addr 172.21.15.110:0/2350249123 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:55.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680d300640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6808102250 0x7f680819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f6807fff640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6808101890 0x7f680819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 -- 172.21.15.110:0/2350249123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6808101890 msgr2=0x7f680819c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6808101890 0x7f680819c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 -- 172.21.15.110:0/2350249123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6808102250 msgr2=0x7f680819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:55.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6808102250 0x7f680819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:55.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 -- 172.21.15.110:0/2350249123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680819f5a0 con 0x7f68081006a0 2024-04-25T00:01:55.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f6807fff640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6808101890 0x7f680819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:55.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680d300640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6808102250 0x7f680819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:55.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680caff640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 0x7f680806a970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009230 tx=0x7f67fc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f6805ffb640 1 -- 172.21.15.110:0/2350249123 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc03f030 con 0x7f68081006a0 2024-04-25T00:01:55.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680819f830 con 0x7f68081006a0 2024-04-25T00:01:55.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f6805ffb640 1 -- 172.21.15.110:0/2350249123 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67fc003cb0 con 0x7f68081006a0 2024-04-25T00:01:55.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.812+0000 7f6805ffb640 1 -- 172.21.15.110:0/2350249123 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc035070 con 0x7f68081006a0 2024-04-25T00:01:55.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.813+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680819fd20 con 0x7f68081006a0 2024-04-25T00:01:55.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.813+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6808107a40 con 0x7f68081006a0 2024-04-25T00:01:55.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.814+0000 7f6805ffb640 1 -- 172.21.15.110:0/2350249123 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f67fc042030 con 0x7f68081006a0 2024-04-25T00:01:55.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.815+0000 7f6805ffb640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f67f80a2b90 0x7f67f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:55.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.815+0000 7f6807fff640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f67f80a2b90 0x7f67f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:55.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.815+0000 7f6805ffb640 1 -- 172.21.15.110:0/2350249123 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f67fc0ef7a0 con 0x7f68081006a0 2024-04-25T00:01:55.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.816+0000 7f6807fff640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f67f80a2b90 0x7f67f80a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f680806a6e0 tx=0x7f67f4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:55.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.820+0000 7f6805ffb640 1 -- 172.21.15.110:0/2350249123 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f67fc0bc6d0 con 0x7f68081006a0 2024-04-25T00:01:55.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.996+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6808062340 con 0x7f68081006a0 2024-04-25T00:01:55.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.997+0000 7f6805ffb640 1 -- 172.21.15.110:0/2350249123 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7f67fc0bbfd0 con 0x7f68081006a0 2024-04-25T00:01:55.998 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:55.998 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f67f80a2b90 msgr2=0x7f67f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f67f80a2b90 0x7f67f80a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f680806a6e0 tx=0x7f67f4005d30 comp rx=0 tx=0).stop 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 msgr2=0x7f680806a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 0x7f680806a970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009230 tx=0x7f67fc004340 comp rx=0 tx=0).stop 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 shutdown_connections 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f67f80a2b90 0x7f67f80a5050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6808102250 0x7f680819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6808101890 0x7f680819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 --2- 172.21.15.110:0/2350249123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68081006a0 0x7f680806a970 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 >> 172.21.15.110:0/2350249123 conn(0x7f68080fbc30 msgr2=0x7f6808104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 shutdown_connections 2024-04-25T00:01:56.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:55.999+0000 7f680ed63640 1 -- 172.21.15.110:0/2350249123 wait complete. 2024-04-25T00:01:56.009 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:55 smithi110 ceph-mon[29187]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:56.010 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:55 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3014115132' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:01:56.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:01.224042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:02.090769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:56.011 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:55 smithi188 ceph-mon[33285]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:55 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3014115132' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:01:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:55 smithi173 ceph-mon[32402]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:01:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:55 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3014115132' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:01:56.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.111+0000 7fa1adfe3640 1 -- 172.21.15.110:0/1099072360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 msgr2=0x7fa1a8101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.111+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/1099072360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 0x7fa1a8101bb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa198009ca0 tx=0x7fa19802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.111+0000 7fa1adfe3640 1 -- 172.21.15.110:0/1099072360 shutdown_connections 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.111+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/1099072360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1a8103340 0x7fa1a8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.111+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/1099072360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a8102980 0x7fa1a8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.111+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/1099072360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 0x7fa1a8101bb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.111+0000 7fa1adfe3640 1 -- 172.21.15.110:0/1099072360 >> 172.21.15.110:0/1099072360 conn(0x7fa1a80fcd20 msgr2=0x7fa1a80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 -- 172.21.15.110:0/1099072360 shutdown_connections 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 -- 172.21.15.110:0/1099072360 wait complete. 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 Processor -- start 2024-04-25T00:01:56.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 -- start start 2024-04-25T00:01:56.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 0x7fa1a8197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1a8102980 0x7fa1a8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a8103340 0x7fa1a819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810d2b0 con 0x7fa1a8101790 2024-04-25T00:01:56.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810d130 con 0x7fa1a8102980 2024-04-25T00:01:56.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1adfe3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810d430 con 0x7fa1a8103340 2024-04-25T00:01:56.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 0x7fa1a8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 0x7fa1a8197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40878/0 (socket says 172.21.15.110:40878) 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.112+0000 7fa1a77fe640 1 -- 172.21.15.110:0/4113352337 learned_addr learned my addr 172.21.15.110:0/4113352337 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a7fff640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a8103340 0x7fa1a819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa19effd640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1a8102980 0x7fa1a8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a77fe640 1 -- 172.21.15.110:0/4113352337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a8103340 msgr2=0x7fa1a819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a77fe640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a8103340 0x7fa1a819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a77fe640 1 -- 172.21.15.110:0/4113352337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1a8102980 msgr2=0x7fa1a8197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a77fe640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1a8102980 0x7fa1a8197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a77fe640 1 -- 172.21.15.110:0/4113352337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a819f5a0 con 0x7fa1a8101790 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a7fff640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a8103340 0x7fa1a819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a77fe640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 0x7fa1a8197980 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa198003ba0 tx=0x7fa198003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:56.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a57fa640 1 -- 172.21.15.110:0/4113352337 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198004040 con 0x7fa1a8101790 2024-04-25T00:01:56.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a57fa640 1 -- 172.21.15.110:0/4113352337 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa198004690 con 0x7fa1a8101790 2024-04-25T00:01:56.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a819f830 con 0x7fa1a8101790 2024-04-25T00:01:56.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1a57fa640 1 -- 172.21.15.110:0/4113352337 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198030cd0 con 0x7fa1a8101790 2024-04-25T00:01:56.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.113+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a819fd80 con 0x7fa1a8101790 2024-04-25T00:01:56.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.114+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a8108b80 con 0x7fa1a8101790 2024-04-25T00:01:56.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.115+0000 7fa1a57fa640 1 -- 172.21.15.110:0/4113352337 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa1980041e0 con 0x7fa1a8101790 2024-04-25T00:01:56.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.116+0000 7fa1a57fa640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa1940a2b90 0x7fa1940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.116+0000 7fa19effd640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa1940a2b90 0x7fa1940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.116+0000 7fa1a57fa640 1 -- 172.21.15.110:0/4113352337 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa1980f0610 con 0x7fa1a8101790 2024-04-25T00:01:56.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.116+0000 7fa19effd640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa1940a2b90 0x7fa1940a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8198e90 tx=0x7fa190005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:56.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.120+0000 7fa1a57fa640 1 -- 172.21.15.110:0/4113352337 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa1980bd5a0 con 0x7fa1a8101790 2024-04-25T00:01:56.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.296+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1a8062340 con 0x7fa1a8101790 2024-04-25T00:01:56.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.297+0000 7fa1a57fa640 1 -- 172.21.15.110:0/4113352337 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fa1980bcea0 con 0x7fa1a8101790 2024-04-25T00:01:56.297 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T00:01:56.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa1940a2b90 msgr2=0x7fa1940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa1940a2b90 0x7fa1940a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8198e90 tx=0x7fa190005f90 comp rx=0 tx=0).stop 2024-04-25T00:01:56.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 msgr2=0x7fa1a8197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 0x7fa1a8197980 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa198003ba0 tx=0x7fa198003bd0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 shutdown_connections 2024-04-25T00:01:56.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa1940a2b90 0x7fa1940a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a8103340 0x7fa1a819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1a8102980 0x7fa1a8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 --2- 172.21.15.110:0/4113352337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8101790 0x7fa1a8197980 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 >> 172.21.15.110:0/4113352337 conn(0x7fa1a80fcd20 msgr2=0x7fa1a8105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:56.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 shutdown_connections 2024-04-25T00:01:56.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.299+0000 7fa1adfe3640 1 -- 172.21.15.110:0/4113352337 wait complete. 2024-04-25T00:01:56.310 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:01:56.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 -- 172.21.15.110:0/4130909164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 msgr2=0x7fd038102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 --2- 172.21.15.110:0/4130909164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 0x7fd038102130 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd028009900 tx=0x7fd02802a400 comp rx=0 tx=0).stop 2024-04-25T00:01:56.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 -- 172.21.15.110:0/4130909164 shutdown_connections 2024-04-25T00:01:56.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 --2- 172.21.15.110:0/4130909164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd038101290 0x7fd038108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 --2- 172.21.15.110:0/4130909164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0381008d0 0x7fd038100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 --2- 172.21.15.110:0/4130909164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 0x7fd038102130 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 -- 172.21.15.110:0/4130909164 >> 172.21.15.110:0/4130909164 conn(0x7fd0380fb380 msgr2=0x7fd0380fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 -- 172.21.15.110:0/4130909164 shutdown_connections 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 -- 172.21.15.110:0/4130909164 wait complete. 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 Processor -- start 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.410+0000 7fd03e17e640 1 -- start start 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd03e17e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0381008d0 0x7fd0381979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd03e17e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd038101290 0x7fd038197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd03e17e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 0x7fd03819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd03e17e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810c230 con 0x7fd038101d10 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd03e17e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810c0b0 con 0x7fd0381008d0 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd03e17e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd03810c3b0 con 0x7fd038101290 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 0x7fd03819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 0x7fd03819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40892/0 (socket says 172.21.15.110:40892) 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 -- 172.21.15.110:0/1097314355 learned_addr learned my addr 172.21.15.110:0/1097314355 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:56.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd0377fe640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0381008d0 0x7fd0381979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd02effd640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd038101290 0x7fd038197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 -- 172.21.15.110:0/1097314355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd038101290 msgr2=0x7fd038197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd038101290 0x7fd038197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 -- 172.21.15.110:0/1097314355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0381008d0 msgr2=0x7fd0381979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0381008d0 0x7fd0381979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 -- 172.21.15.110:0/1097314355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03819f5f0 con 0x7fd038101d10 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd02effd640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd038101290 0x7fd038197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd037fff640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 0x7fd03819ef80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd03000b970 tx=0x7fd03000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd0357fa640 1 -- 172.21.15.110:0/1097314355 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030019070 con 0x7fd038101d10 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd0357fa640 1 -- 172.21.15.110:0/1097314355 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd030014440 con 0x7fd038101d10 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03819f8e0 con 0x7fd038101d10 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd0357fa640 1 -- 172.21.15.110:0/1097314355 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0300075d0 con 0x7fd038101d10 2024-04-25T00:01:56.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.411+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03819feb0 con 0x7fd038101d10 2024-04-25T00:01:56.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.412+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0381a0140 con 0x7fd038101d10 2024-04-25T00:01:56.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.414+0000 7fd0357fa640 1 -- 172.21.15.110:0/1097314355 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd03001c070 con 0x7fd038101d10 2024-04-25T00:01:56.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.414+0000 7fd0357fa640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0240a2c10 0x7fd0240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.414+0000 7fd0377fe640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0240a2c10 0x7fd0240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.414+0000 7fd0357fa640 1 -- 172.21.15.110:0/1097314355 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd0300ce9c0 con 0x7fd038101d10 2024-04-25T00:01:56.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.415+0000 7fd0377fe640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0240a2c10 0x7fd0240a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd028000c00 tx=0x7fd028005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:56.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.418+0000 7fd0357fa640 1 -- 172.21.15.110:0/1097314355 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd03009b300 con 0x7fd038101d10 2024-04-25T00:01:56.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.595+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd0381a0310 con 0x7fd038101d10 2024-04-25T00:01:56.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.595+0000 7fd0357fa640 1 -- 172.21.15.110:0/1097314355 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fd03009f600 con 0x7fd038101d10 2024-04-25T00:01:56.596 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T00:01:56.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0240a2c10 msgr2=0x7fd0240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0240a2c10 0x7fd0240a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd028000c00 tx=0x7fd028005e40 comp rx=0 tx=0).stop 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 msgr2=0x7fd03819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 0x7fd03819ef80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd03000b970 tx=0x7fd03000be30 comp rx=0 tx=0).stop 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 shutdown_connections 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd0240a2c10 0x7fd0240a50d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd038101d10 0x7fd03819ef80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd038101290 0x7fd038197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 --2- 172.21.15.110:0/1097314355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0381008d0 0x7fd0381979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 >> 172.21.15.110:0/1097314355 conn(0x7fd0380fb380 msgr2=0x7fd0380ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 shutdown_connections 2024-04-25T00:01:56.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.597+0000 7fd03e17e640 1 -- 172.21.15.110:0/1097314355 wait complete. 2024-04-25T00:01:56.609 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.708+0000 7f80b5f7e640 1 -- 172.21.15.110:0/2612325605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0102990 msgr2=0x7f80b0102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.708+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/2612325605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0102990 0x7f80b0102e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009900 tx=0x7f80a402a410 comp rx=0 tx=0).stop 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 -- 172.21.15.110:0/2612325605 shutdown_connections 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/2612325605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b0103350 0x7f80b0109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/2612325605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0102990 0x7f80b0102e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/2612325605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80b01017a0 0x7f80b0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 -- 172.21.15.110:0/2612325605 >> 172.21.15.110:0/2612325605 conn(0x7f80b00fcd10 msgr2=0x7f80b00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 -- 172.21.15.110:0/2612325605 shutdown_connections 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 -- 172.21.15.110:0/2612325605 wait complete. 2024-04-25T00:01:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 Processor -- start 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 -- start start 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80b01017a0 0x7f80b00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b0102990 0x7f80b0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0103350 0x7f80b0074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010d390 con 0x7f80b0103350 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010d210 con 0x7f80b0102990 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.709+0000 7f80b5f7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010d510 con 0x7f80b01017a0 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0103350 0x7f80b0074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0103350 0x7f80b0074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35172/0 (socket says 172.21.15.110:35172) 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 -- 172.21.15.110:0/333123780 learned_addr learned my addr 172.21.15.110:0/333123780 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80af7fe640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80b01017a0 0x7f80b00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80aeffd640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b0102990 0x7f80b0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 -- 172.21.15.110:0/333123780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80b01017a0 msgr2=0x7f80b00754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80b01017a0 0x7f80b00754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 -- 172.21.15.110:0/333123780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b0102990 msgr2=0x7f80b0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b0102990 0x7f80b0073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 -- 172.21.15.110:0/333123780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b0074900 con 0x7f80b0103350 2024-04-25T00:01:56.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.710+0000 7f80affff640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0103350 0x7f80b0074070 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f80a800eaa0 tx=0x7f80a800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:56.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.711+0000 7f80acff9640 1 -- 172.21.15.110:0/333123780 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8004d20 con 0x7f80b0103350 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.711+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b01a1b80 con 0x7f80b0103350 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.711+0000 7f80acff9640 1 -- 172.21.15.110:0/333123780 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80a8005020 con 0x7f80b0103350 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.711+0000 7f80acff9640 1 -- 172.21.15.110:0/333123780 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80116b0 con 0x7f80b0103350 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.711+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b01a2050 con 0x7f80b0103350 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.712+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b00689b0 con 0x7f80b0103350 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.713+0000 7f80acff9640 1 -- 172.21.15.110:0/333123780 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f80a80051d0 con 0x7f80b0103350 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.714+0000 7f80acff9640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f80980a2b90 0x7f80980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.714+0000 7f80af7fe640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f80980a2b90 0x7f80980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.714+0000 7f80acff9640 1 -- 172.21.15.110:0/333123780 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f80a80cf210 con 0x7f80b0103350 2024-04-25T00:01:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.714+0000 7f80af7fe640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f80980a2b90 0x7f80980a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f80b01027f0 tx=0x7f80a0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:56.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.721+0000 7f80acff9640 1 -- 172.21.15.110:0/333123780 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f80a8016030 con 0x7f80b0103350 2024-04-25T00:01:56.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.898+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f80b0062340 con 0x7f80b0103350 2024-04-25T00:01:56.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.898+0000 7f80acff9640 1 -- 172.21.15.110:0/333123780 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f80a809c1a0 con 0x7f80b0103350 2024-04-25T00:01:56.899 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 64 2024-04-25T00:01:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f80980a2b90 msgr2=0x7f80980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f80980a2b90 0x7f80980a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f80b01027f0 tx=0x7f80a0009110 comp rx=0 tx=0).stop 2024-04-25T00:01:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0103350 msgr2=0x7f80b0074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0103350 0x7f80b0074070 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f80a800eaa0 tx=0x7f80a800eea0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 shutdown_connections 2024-04-25T00:01:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f80980a2b90 0x7f80980a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80b0103350 0x7f80b0074070 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80b0102990 0x7f80b0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 --2- 172.21.15.110:0/333123780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80b01017a0 0x7f80b00754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 >> 172.21.15.110:0/333123780 conn(0x7f80b00fcd10 msgr2=0x7f80b0105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 shutdown_connections 2024-04-25T00:01:56.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:56.901+0000 7f80b5f7e640 1 -- 172.21.15.110:0/333123780 wait complete. 2024-04-25T00:01:56.911 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:56 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2350249123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:56.911 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:56 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4113352337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:56.911 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:56 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1097314355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:56.912 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 -- 172.21.15.110:0/2275035569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 msgr2=0x7f6680100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 --2- 172.21.15.110:0/2275035569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 0x7f6680100ac0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6678009900 tx=0x7f667802a410 comp rx=0 tx=0).stop 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 -- 172.21.15.110:0/2275035569 shutdown_connections 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 --2- 172.21.15.110:0/2275035569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6680102250 0x7f6680108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 --2- 172.21.15.110:0/2275035569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6680101890 0x7f6680101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 --2- 172.21.15.110:0/2275035569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 0x7f6680100ac0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 -- 172.21.15.110:0/2275035569 >> 172.21.15.110:0/2275035569 conn(0x7f66800fbc30 msgr2=0x7f66800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 -- 172.21.15.110:0/2275035569 shutdown_connections 2024-04-25T00:01:57.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.009+0000 7f6684d70640 1 -- 172.21.15.110:0/2275035569 wait complete. 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f6684d70640 1 Processor -- start 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f6684d70640 1 -- start start 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f6684d70640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 0x7f66801979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f6684d70640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6680101890 0x7f6680197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f6684d70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6680102250 0x7f668019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f6684d70640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f668010d2e0 con 0x7f66801006a0 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f6684d70640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f668010d160 con 0x7f6680101890 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f6684d70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f668010d460 con 0x7f6680102250 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 0x7f66801979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 0x7f66801979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35194/0 (socket says 172.21.15.110:35194) 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 -- 172.21.15.110:0/3547237484 learned_addr learned my addr 172.21.15.110:0/3547237484 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667dd9b640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6680101890 0x7f6680197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667ed9d640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6680102250 0x7f668019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 -- 172.21.15.110:0/3547237484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6680102250 msgr2=0x7f668019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6680102250 0x7f668019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 -- 172.21.15.110:0/3547237484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6680101890 msgr2=0x7f6680197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6680101890 0x7f6680197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 -- 172.21.15.110:0/3547237484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668019f5e0 con 0x7f66801006a0 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667ed9d640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6680102250 0x7f668019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.010+0000 7f667e59c640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 0x7f66801979c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f66780314e0 tx=0x7f6678031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.011+0000 7f666f7fe640 1 -- 172.21.15.110:0/3547237484 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667803f030 con 0x7f66801006a0 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.011+0000 7f666f7fe640 1 -- 172.21.15.110:0/3547237484 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66780044b0 con 0x7f66801006a0 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.011+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668019f870 con 0x7f66801006a0 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.011+0000 7f666f7fe640 1 -- 172.21.15.110:0/3547237484 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678035070 con 0x7f66801006a0 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.011+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668019fd60 con 0x7f66801006a0 2024-04-25T00:01:57.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.012+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6680107a30 con 0x7f66801006a0 2024-04-25T00:01:57.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.013+0000 7f666f7fe640 1 -- 172.21.15.110:0/3547237484 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6678042030 con 0x7f66801006a0 2024-04-25T00:01:57.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.013+0000 7f666f7fe640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66680a2b90 0x7f66680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.013+0000 7f667dd9b640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66680a2b90 0x7f66680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.013+0000 7f666f7fe640 1 -- 172.21.15.110:0/3547237484 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f66780ef5d0 con 0x7f66801006a0 2024-04-25T00:01:57.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.014+0000 7f667dd9b640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66680a2b90 0x7f66680a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6680198ed0 tx=0x7f6674009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:56 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2350249123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:56 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4113352337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:56 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1097314355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:57.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.018+0000 7f666f7fe640 1 -- 172.21.15.110:0/3547237484 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f66780bc560 con 0x7f66801006a0 2024-04-25T00:01:57.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:56 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2350249123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:57.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:56 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4113352337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:57.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:56 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1097314355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:57.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.230+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f668002d170 con 0x7f66801006a0 2024-04-25T00:01:57.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.231+0000 7f666f7fe640 1 -- 172.21.15.110:0/3547237484 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7f66780bbe60 con 0x7f66801006a0 2024-04-25T00:01:57.231 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:57.231 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:00:02.856564+0000","modified":"2024-04-25T00:00:07.553801+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24457,24476,24512]},"id":1}]} 2024-04-25T00:01:57.232 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-04-25T00:01:57.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66680a2b90 msgr2=0x7f66680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66680a2b90 0x7f66680a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6680198ed0 tx=0x7f6674009760 comp rx=0 tx=0).stop 2024-04-25T00:01:57.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 msgr2=0x7f66801979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 0x7f66801979c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f66780314e0 tx=0x7f6678031a30 comp rx=0 tx=0).stop 2024-04-25T00:01:57.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 shutdown_connections 2024-04-25T00:01:57.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66680a2b90 0x7f66680a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6680102250 0x7f668019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6680101890 0x7f6680197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 --2- 172.21.15.110:0/3547237484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66801006a0 0x7f66801979c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 >> 172.21.15.110:0/3547237484 conn(0x7f66800fbc30 msgr2=0x7f6680104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:57.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 shutdown_connections 2024-04-25T00:01:57.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.233+0000 7f6684d70640 1 -- 172.21.15.110:0/3547237484 wait complete. 2024-04-25T00:01:57.245 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 -- 172.21.15.110:0/1743905463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029a0 msgr2=0x7fa530102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 --2- 172.21.15.110:0/1743905463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029a0 0x7fa530102e20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa524009900 tx=0x7fa52402a410 comp rx=0 tx=0).stop 2024-04-25T00:01:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 -- 172.21.15.110:0/1743905463 shutdown_connections 2024-04-25T00:01:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 --2- 172.21.15.110:0/1743905463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa530103360 0x7fa530109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 --2- 172.21.15.110:0/1743905463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029a0 0x7fa530102e20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 --2- 172.21.15.110:0/1743905463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5301017b0 0x7fa530101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 -- 172.21.15.110:0/1743905463 >> 172.21.15.110:0/1743905463 conn(0x7fa5300fcd40 msgr2=0x7fa5300ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 -- 172.21.15.110:0/1743905463 shutdown_connections 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.347+0000 7fa53616a640 1 -- 172.21.15.110:0/1743905463 wait complete. 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa53616a640 1 Processor -- start 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa53616a640 1 -- start start 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa53616a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301017b0 0x7fa5300754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa53616a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5301029a0 0x7fa530073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa53616a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa530103360 0x7fa530074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa53616a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010d350 con 0x7fa5301017b0 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa53616a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010d1d0 con 0x7fa5301029a0 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa53616a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010d4d0 con 0x7fa530103360 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301017b0 0x7fa5300754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301017b0 0x7fa5300754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35216/0 (socket says 172.21.15.110:35216) 2024-04-25T00:01:57.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 -- 172.21.15.110:0/4157818378 learned_addr learned my addr 172.21.15.110:0/4157818378 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52ffff640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa530103360 0x7fa530074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 -- 172.21.15.110:0/4157818378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa530103360 msgr2=0x7fa530074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52effd640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5301029a0 0x7fa530073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa530103360 0x7fa530074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 -- 172.21.15.110:0/4157818378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5301029a0 msgr2=0x7fa530073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5301029a0 0x7fa530073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 -- 172.21.15.110:0/4157818378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa530074900 con 0x7fa5301017b0 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52ffff640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa530103360 0x7fa530074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52effd640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5301029a0 0x7fa530073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.348+0000 7fa52f7fe640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301017b0 0x7fa5300754f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b450 tx=0x7fa52000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.349+0000 7fa52cff9640 1 -- 172.21.15.110:0/4157818378 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520015020 con 0x7fa5301017b0 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.349+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301a1ba0 con 0x7fa5301017b0 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.349+0000 7fa52cff9640 1 -- 172.21.15.110:0/4157818378 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa520011d40 con 0x7fa5301017b0 2024-04-25T00:01:57.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.349+0000 7fa52cff9640 1 -- 172.21.15.110:0/4157818378 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52000f340 con 0x7fa5301017b0 2024-04-25T00:01:57.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.349+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301a20a0 con 0x7fa5301017b0 2024-04-25T00:01:57.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.350+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa530108b40 con 0x7fa5301017b0 2024-04-25T00:01:57.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.351+0000 7fa52cff9640 1 -- 172.21.15.110:0/4157818378 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa520018020 con 0x7fa5301017b0 2024-04-25T00:01:57.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.351+0000 7fa52cff9640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa5180a2b90 0x7fa5180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.351+0000 7fa52effd640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa5180a2b90 0x7fa5180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.351+0000 7fa52cff9640 1 -- 172.21.15.110:0/4157818378 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa5200cf180 con 0x7fa5301017b0 2024-04-25T00:01:57.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.352+0000 7fa52effd640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa5180a2b90 0x7fa5180a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa524006010 tx=0x7fa524000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:57.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.355+0000 7fa52cff9640 1 -- 172.21.15.110:0/4157818378 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa52001f020 con 0x7fa5301017b0 2024-04-25T00:01:57.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.532+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa53002d3e0 con 0x7fa5301017b0 2024-04-25T00:01:57.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.533+0000 7fa52cff9640 1 -- 172.21.15.110:0/4157818378 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7fa52000d040 con 0x7fa5301017b0 2024-04-25T00:01:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:57.534 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:57.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.535+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa5180a2b90 msgr2=0x7fa5180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.535+0000 7fa53616a640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa5180a2b90 0x7fa5180a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa524006010 tx=0x7fa524000bc0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.535+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301017b0 msgr2=0x7fa5300754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.535+0000 7fa53616a640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301017b0 0x7fa5300754f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b450 tx=0x7fa52000b910 comp rx=0 tx=0).stop 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.536+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 shutdown_connections 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.536+0000 7fa53616a640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa5180a2b90 0x7fa5180a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.536+0000 7fa53616a640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa530103360 0x7fa530074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.536+0000 7fa53616a640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5301029a0 0x7fa530073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.536+0000 7fa53616a640 1 --2- 172.21.15.110:0/4157818378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301017b0 0x7fa5300754f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.536+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 >> 172.21.15.110:0/4157818378 conn(0x7fa5300fcd40 msgr2=0x7fa530105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.536+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 shutdown_connections 2024-04-25T00:01:57.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.536+0000 7fa53616a640 1 -- 172.21.15.110:0/4157818378 wait complete. 2024-04-25T00:01:57.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:01.224042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:02.090769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:01:57.548 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:01:57.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 -- 172.21.15.110:0/2304430781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 msgr2=0x7f594c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 --2- 172.21.15.110:0/2304430781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 0x7f594c1098b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5944009ca0 tx=0x7f594402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 -- 172.21.15.110:0/2304430781 shutdown_connections 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 --2- 172.21.15.110:0/2304430781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 0x7f594c1098b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 --2- 172.21.15.110:0/2304430781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f594c104b30 0x7f594c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 --2- 172.21.15.110:0/2304430781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594c102200 0x7f594c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 -- 172.21.15.110:0/2304430781 >> 172.21.15.110:0/2304430781 conn(0x7f594c0fbc30 msgr2=0x7f594c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 -- 172.21.15.110:0/2304430781 shutdown_connections 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.649+0000 7f5953e12640 1 -- 172.21.15.110:0/2304430781 wait complete. 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5953e12640 1 Processor -- start 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5953e12640 1 -- start start 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5953e12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f594c102200 0x7f594c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5953e12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594c104b30 0x7f594c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5953e12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 0x7f594c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5953e12640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10d2d0 con 0x7f594c107460 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5953e12640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10d150 con 0x7f594c102200 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5953e12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10d450 con 0x7f594c104b30 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 0x7f594c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 0x7f594c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35226/0 (socket says 172.21.15.110:35226) 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 -- 172.21.15.110:0/901035227 learned_addr learned my addr 172.21.15.110:0/901035227 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59513ad640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594c104b30 0x7f594c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5951bae640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f594c102200 0x7f594c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 -- 172.21.15.110:0/901035227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594c104b30 msgr2=0x7f594c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594c104b30 0x7f594c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 -- 172.21.15.110:0/901035227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f594c102200 msgr2=0x7f594c197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f594c102200 0x7f594c197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 -- 172.21.15.110:0/901035227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c19f5a0 con 0x7f594c107460 2024-04-25T00:01:57.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f59523af640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 0x7f594c19ef30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5944031970 tx=0x7f5944003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:57.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.650+0000 7f5942ffd640 1 -- 172.21.15.110:0/901035227 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944004040 con 0x7f594c107460 2024-04-25T00:01:57.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.651+0000 7f5942ffd640 1 -- 172.21.15.110:0/901035227 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5944004690 con 0x7f594c107460 2024-04-25T00:01:57.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.651+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594c19f830 con 0x7f594c107460 2024-04-25T00:01:57.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.651+0000 7f5942ffd640 1 -- 172.21.15.110:0/901035227 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944030c60 con 0x7f594c107460 2024-04-25T00:01:57.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.651+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594c19fd80 con 0x7f594c107460 2024-04-25T00:01:57.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.652+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594c0fd720 con 0x7f594c107460 2024-04-25T00:01:57.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.653+0000 7f5942ffd640 1 -- 172.21.15.110:0/901035227 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f59440041e0 con 0x7f594c107460 2024-04-25T00:01:57.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.653+0000 7f5942ffd640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59380a2b90 0x7f59380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.653+0000 7f5951bae640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59380a2b90 0x7f59380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.653+0000 7f5942ffd640 1 -- 172.21.15.110:0/901035227 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f59440f0710 con 0x7f594c107460 2024-04-25T00:01:57.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.654+0000 7f5951bae640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59380a2b90 0x7f59380a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5948009aa0 tx=0x7f5948005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:57.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.658+0000 7f5942ffd640 1 -- 172.21.15.110:0/901035227 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59440bd6a0 con 0x7f594c107460 2024-04-25T00:01:57.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.835+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f594c062340 con 0x7f594c107460 2024-04-25T00:01:57.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.836+0000 7f5942ffd640 1 -- 172.21.15.110:0/901035227 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f59440bcfa0 con 0x7f594c107460 2024-04-25T00:01:57.836 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T00:01:57.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59380a2b90 msgr2=0x7f59380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59380a2b90 0x7f59380a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5948009aa0 tx=0x7f5948005f90 comp rx=0 tx=0).stop 2024-04-25T00:01:57.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 msgr2=0x7f594c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 0x7f594c19ef30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5944031970 tx=0x7f5944003ba0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 shutdown_connections 2024-04-25T00:01:57.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59380a2b90 0x7f59380a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f594c107460 0x7f594c19ef30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f594c104b30 0x7f594c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 --2- 172.21.15.110:0/901035227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f594c102200 0x7f594c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 >> 172.21.15.110:0/901035227 conn(0x7f594c0fbc30 msgr2=0x7f594c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:57.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 shutdown_connections 2024-04-25T00:01:57.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.838+0000 7f5953e12640 1 -- 172.21.15.110:0/901035227 wait complete. 2024-04-25T00:01:57.848 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:57 smithi110 ceph-mon[29187]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:57.848 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:57 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/333123780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:57.848 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:57 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3547237484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:57.848 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:57 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4157818378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:57.849 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:01:57.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 -- 172.21.15.110:0/3867352109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c101870 msgr2=0x7f728c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 --2- 172.21.15.110:0/3867352109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c101870 0x7f728c101cf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7280009900 tx=0x7f728002a410 comp rx=0 tx=0).stop 2024-04-25T00:01:57.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 -- 172.21.15.110:0/3867352109 shutdown_connections 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 --2- 172.21.15.110:0/3867352109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c102230 0x7f728c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 --2- 172.21.15.110:0/3867352109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c101870 0x7f728c101cf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 --2- 172.21.15.110:0/3867352109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f728c100680 0x7f728c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 -- 172.21.15.110:0/3867352109 >> 172.21.15.110:0/3867352109 conn(0x7f728c0fbbf0 msgr2=0x7f728c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 -- 172.21.15.110:0/3867352109 shutdown_connections 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.949+0000 7f7291309640 1 -- 172.21.15.110:0/3867352109 wait complete. 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f7291309640 1 Processor -- start 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f7291309640 1 -- start start 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f7291309640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c100680 0x7f728c196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f7291309640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c101870 0x7f728c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f7291309640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f728c102230 0x7f728c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f7291309640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f728c10c190 con 0x7f728c100680 2024-04-25T00:01:57.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f7291309640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f728c10c010 con 0x7f728c102230 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f7291309640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f728c10c310 con 0x7f728c101870 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f728affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c100680 0x7f728c196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f728affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c100680 0x7f728c196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35246/0 (socket says 172.21.15.110:35246) 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f728affd640 1 -- 172.21.15.110:0/162272221 learned_addr learned my addr 172.21.15.110:0/162272221 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.950+0000 7f728a7fc640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c101870 0x7f728c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f728affd640 1 -- 172.21.15.110:0/162272221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c101870 msgr2=0x7f728c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f728affd640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c101870 0x7f728c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f728affd640 1 -- 172.21.15.110:0/162272221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f728c102230 msgr2=0x7f728c19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f728affd640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f728c102230 0x7f728c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f728affd640 1 -- 172.21.15.110:0/162272221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c19e490 con 0x7f728c100680 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f728affd640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c100680 0x7f728c196870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f727c00ccb0 tx=0x7f727c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f7273fff640 1 -- 172.21.15.110:0/162272221 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c003990 con 0x7f728c100680 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f7273fff640 1 -- 172.21.15.110:0/162272221 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727c004020 con 0x7f728c100680 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c19e780 con 0x7f728c100680 2024-04-25T00:01:57.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f7273fff640 1 -- 172.21.15.110:0/162272221 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c00f6b0 con 0x7f728c100680 2024-04-25T00:01:57.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.951+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c19ecd0 con 0x7f728c100680 2024-04-25T00:01:57.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.952+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c107a20 con 0x7f728c100680 2024-04-25T00:01:57.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.953+0000 7f7273fff640 1 -- 172.21.15.110:0/162272221 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f727c003b30 con 0x7f728c100680 2024-04-25T00:01:57.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.954+0000 7f7273fff640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f72740a2b90 0x7f72740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:57.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.954+0000 7f728a7fc640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f72740a2b90 0x7f72740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:57.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.954+0000 7f7273fff640 1 -- 172.21.15.110:0/162272221 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f727c0cf490 con 0x7f728c100680 2024-04-25T00:01:57.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.954+0000 7f728a7fc640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f72740a2b90 0x7f72740a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f728c197d80 tx=0x7f72800314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:57.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:57.958+0000 7f7273fff640 1 -- 172.21.15.110:0/162272221 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f727c014030 con 0x7f728c100680 2024-04-25T00:01:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:57 smithi188 ceph-mon[33285]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:57 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/333123780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:57 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3547237484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:57 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4157818378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:57 smithi173 ceph-mon[32402]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:01:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:57 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/333123780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:57 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3547237484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:01:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:57 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4157818378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:01:58.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.133+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f728c0623e0 con 0x7f728c100680 2024-04-25T00:01:58.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.133+0000 7f7273fff640 1 -- 172.21.15.110:0/162272221 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f727c09c420 con 0x7f728c100680 2024-04-25T00:01:58.133 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T00:01:58.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f72740a2b90 msgr2=0x7f72740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f72740a2b90 0x7f72740a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f728c197d80 tx=0x7f72800314a0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c100680 msgr2=0x7f728c196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c100680 0x7f728c196870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f727c00ccb0 tx=0x7f727c00a620 comp rx=0 tx=0).stop 2024-04-25T00:01:58.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 shutdown_connections 2024-04-25T00:01:58.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f72740a2b90 0x7f72740a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f728c102230 0x7f728c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f728c101870 0x7f728c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 --2- 172.21.15.110:0/162272221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f728c100680 0x7f728c196870 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 >> 172.21.15.110:0/162272221 conn(0x7f728c0fbbf0 msgr2=0x7f728c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:58.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 shutdown_connections 2024-04-25T00:01:58.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.135+0000 7f7291309640 1 -- 172.21.15.110:0/162272221 wait complete. 2024-04-25T00:01:58.147 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 -- 172.21.15.110:0/1605126108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18102250 msgr2=0x7f1f18108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/1605126108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18102250 0x7f1f18108ae0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009ca0 tx=0x7f1f1002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 -- 172.21.15.110:0/1605126108 shutdown_connections 2024-04-25T00:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/1605126108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18102250 0x7f1f18108ae0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/1605126108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f18101890 0x7f1f18101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/1605126108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f181006a0 0x7f1f18100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 -- 172.21.15.110:0/1605126108 >> 172.21.15.110:0/1605126108 conn(0x7f1f180fbc30 msgr2=0x7f1f180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 -- 172.21.15.110:0/1605126108 shutdown_connections 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.249+0000 7f1f1cc88640 1 -- 172.21.15.110:0/1605126108 wait complete. 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1cc88640 1 Processor -- start 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1cc88640 1 -- start start 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1cc88640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f181006a0 0x7f1f181979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1cc88640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18101890 0x7f1f18197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1cc88640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f18102250 0x7f1f1819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1cc88640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1810d330 con 0x7f1f18101890 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1cc88640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1810d1b0 con 0x7f1f181006a0 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1cc88640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1810d4b0 con 0x7f1f18102250 2024-04-25T00:01:58.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18101890 0x7f1f18197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18101890 0x7f1f18197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35276/0 (socket says 172.21.15.110:35276) 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 -- 172.21.15.110:0/2573870395 learned_addr learned my addr 172.21.15.110:0/2573870395 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f16d9d640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f18102250 0x7f1f1819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f1659c640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f181006a0 0x7f1f181979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 -- 172.21.15.110:0/2573870395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f18102250 msgr2=0x7f1f1819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f18102250 0x7f1f1819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 -- 172.21.15.110:0/2573870395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f181006a0 msgr2=0x7f1f181979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f181006a0 0x7f1f181979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 -- 172.21.15.110:0/2573870395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1819f610 con 0x7f1f18101890 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f16d9d640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f18102250 0x7f1f1819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f15d9b640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18101890 0x7f1f18197f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b4a0 tx=0x7f1f0800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f077fe640 1 -- 172.21.15.110:0/2573870395 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08014070 con 0x7f1f18101890 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f077fe640 1 -- 172.21.15.110:0/2573870395 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f08010ac0 con 0x7f1f18101890 2024-04-25T00:01:58.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.250+0000 7f1f077fe640 1 -- 172.21.15.110:0/2573870395 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08017c30 con 0x7f1f18101890 2024-04-25T00:01:58.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.251+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1819f900 con 0x7f1f18101890 2024-04-25T00:01:58.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.251+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1819fe00 con 0x7f1f18101890 2024-04-25T00:01:58.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.251+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f18107a90 con 0x7f1f18101890 2024-04-25T00:01:58.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.253+0000 7f1f077fe640 1 -- 172.21.15.110:0/2573870395 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1f08017070 con 0x7f1f18101890 2024-04-25T00:01:58.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.253+0000 7f1f077fe640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f000a2b90 0x7f1f000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.253+0000 7f1f1659c640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f000a2b90 0x7f1f000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.253+0000 7f1f077fe640 1 -- 172.21.15.110:0/2573870395 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1f080ce8a0 con 0x7f1f18101890 2024-04-25T00:01:58.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.254+0000 7f1f1659c640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f000a2b90 0x7f1f000a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005fd0 tx=0x7f1f0c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:58.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.257+0000 7f1f077fe640 1 -- 172.21.15.110:0/2573870395 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f0809b830 con 0x7f1f18101890 2024-04-25T00:01:58.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.431+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f1802d300 con 0x7f1f18101890 2024-04-25T00:01:58.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.431+0000 7f1f077fe640 1 -- 172.21.15.110:0/2573870395 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f1f0809b130 con 0x7f1f18101890 2024-04-25T00:01:58.432 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 64 2024-04-25T00:01:58.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f000a2b90 msgr2=0x7f1f000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f000a2b90 0x7f1f000a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005fd0 tx=0x7f1f0c005850 comp rx=0 tx=0).stop 2024-04-25T00:01:58.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18101890 msgr2=0x7f1f18197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18101890 0x7f1f18197f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b4a0 tx=0x7f1f0800b960 comp rx=0 tx=0).stop 2024-04-25T00:01:58.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 shutdown_connections 2024-04-25T00:01:58.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1f000a2b90 0x7f1f000a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f18102250 0x7f1f1819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f18101890 0x7f1f18197f30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 --2- 172.21.15.110:0/2573870395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f181006a0 0x7f1f181979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 >> 172.21.15.110:0/2573870395 conn(0x7f1f180fbc30 msgr2=0x7f1f18104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:58.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 shutdown_connections 2024-04-25T00:01:58.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.433+0000 7f1f1cc88640 1 -- 172.21.15.110:0/2573870395 wait complete. 2024-04-25T00:01:58.445 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 -- 172.21.15.110:0/3221684560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2778101890 msgr2=0x7f2778101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 --2- 172.21.15.110:0/3221684560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2778101890 0x7f2778101d10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2768009900 tx=0x7f276802a410 comp rx=0 tx=0).stop 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 -- 172.21.15.110:0/3221684560 shutdown_connections 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 --2- 172.21.15.110:0/3221684560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2778102250 0x7f2778108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 --2- 172.21.15.110:0/3221684560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2778101890 0x7f2778101d10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 --2- 172.21.15.110:0/3221684560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27781006a0 0x7f2778100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 -- 172.21.15.110:0/3221684560 >> 172.21.15.110:0/3221684560 conn(0x7f27780fbc30 msgr2=0x7f27780fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 -- 172.21.15.110:0/3221684560 shutdown_connections 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 -- 172.21.15.110:0/3221684560 wait complete. 2024-04-25T00:01:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 Processor -- start 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.544+0000 7f277f789640 1 -- start start 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277f789640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27781006a0 0x7f2778197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277f789640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2778101890 0x7f2778197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277f789640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2778102250 0x7f277819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277f789640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f277810c1d0 con 0x7f27781006a0 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277f789640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f277810c050 con 0x7f2778102250 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277f789640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f277810c350 con 0x7f2778101890 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27781006a0 0x7f2778197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27781006a0 0x7f2778197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35286/0 (socket says 172.21.15.110:35286) 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 -- 172.21.15.110:0/888007332 learned_addr learned my addr 172.21.15.110:0/888007332 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277cd24640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2778101890 0x7f2778197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277dd26640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2778102250 0x7f277819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 -- 172.21.15.110:0/888007332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2778101890 msgr2=0x7f2778197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2778101890 0x7f2778197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 -- 172.21.15.110:0/888007332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2778102250 msgr2=0x7f277819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2778102250 0x7f277819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 -- 172.21.15.110:0/888007332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277819f630 con 0x7f27781006a0 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277cd24640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2778101890 0x7f2778197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277d525640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27781006a0 0x7f2778197a60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f277000eaa0 tx=0x7f277000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f276e7fc640 1 -- 172.21.15.110:0/888007332 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770016070 con 0x7f27781006a0 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f276e7fc640 1 -- 172.21.15.110:0/888007332 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2770003e20 con 0x7f27781006a0 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f276e7fc640 1 -- 172.21.15.110:0/888007332 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700116d0 con 0x7f27781006a0 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277819f920 con 0x7f27781006a0 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.545+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277819fe20 con 0x7f27781006a0 2024-04-25T00:01:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.546+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27781a00b0 con 0x7f27781006a0 2024-04-25T00:01:58.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.547+0000 7f276e7fc640 1 -- 172.21.15.110:0/888007332 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2770019070 con 0x7f27781006a0 2024-04-25T00:01:58.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.548+0000 7f276e7fc640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27640a2c10 0x7f27640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.548+0000 7f277cd24640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27640a2c10 0x7f27640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.548+0000 7f276e7fc640 1 -- 172.21.15.110:0/888007332 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f27700cf510 con 0x7f27781006a0 2024-04-25T00:01:58.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.548+0000 7f277cd24640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27640a2c10 0x7f27640a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2778198f20 tx=0x7f2768032c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:58.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.552+0000 7f276e7fc640 1 -- 172.21.15.110:0/888007332 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f277009c520 con 0x7f27781006a0 2024-04-25T00:01:58.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.763+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27781a0280 con 0x7f27781006a0 2024-04-25T00:01:58.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.764+0000 7f276e7fc640 1 -- 172.21.15.110:0/888007332 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7f277009be20 con 0x7f27781006a0 2024-04-25T00:01:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:58.765 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:00:02.856564+0000","modified":"2024-04-25T00:00:07.553801+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24457,24476,24512]},"id":1}]} 2024-04-25T00:01:58.765 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-04-25T00:01:58.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27640a2c10 msgr2=0x7f27640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27640a2c10 0x7f27640a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2778198f20 tx=0x7f2768032c40 comp rx=0 tx=0).stop 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27781006a0 msgr2=0x7f2778197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27781006a0 0x7f2778197a60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f277000eaa0 tx=0x7f277000ef60 comp rx=0 tx=0).stop 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 shutdown_connections 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27640a2c10 0x7f27640a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2778102250 0x7f277819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2778101890 0x7f2778197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 --2- 172.21.15.110:0/888007332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27781006a0 0x7f2778197a60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 >> 172.21.15.110:0/888007332 conn(0x7f27780fbc30 msgr2=0x7f27780fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 shutdown_connections 2024-04-25T00:01:58.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.766+0000 7f277f789640 1 -- 172.21.15.110:0/888007332 wait complete. 2024-04-25T00:01:58.778 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:01:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 -- 172.21.15.110:0/750174301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc103340 msgr2=0x7f59fc109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 --2- 172.21.15.110:0/750174301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc103340 0x7f59fc109bd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009ca0 tx=0x7f59f402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 -- 172.21.15.110:0/750174301 shutdown_connections 2024-04-25T00:01:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 --2- 172.21.15.110:0/750174301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc103340 0x7f59fc109bd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 --2- 172.21.15.110:0/750174301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59fc102980 0x7f59fc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 --2- 172.21.15.110:0/750174301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59fc101790 0x7f59fc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 -- 172.21.15.110:0/750174301 >> 172.21.15.110:0/750174301 conn(0x7f59fc0fcd20 msgr2=0x7f59fc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 -- 172.21.15.110:0/750174301 shutdown_connections 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 -- 172.21.15.110:0/750174301 wait complete. 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 Processor -- start 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.876+0000 7f5a01bed640 1 -- start start 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f5a01bed640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc101790 0x7f59fc197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f5a01bed640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59fc102980 0x7f59fc197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f5a01bed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59fc103340 0x7f59fc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f5a01bed640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10d3b0 con 0x7f59fc101790 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f5a01bed640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10d230 con 0x7f59fc102980 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f5a01bed640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10d530 con 0x7f59fc103340 2024-04-25T00:01:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc101790 0x7f59fc197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc101790 0x7f59fc197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35298/0 (socket says 172.21.15.110:35298) 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 -- 172.21.15.110:0/4076628753 learned_addr learned my addr 172.21.15.110:0/4076628753 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59faffd640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59fc102980 0x7f59fc197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fbfff640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59fc103340 0x7f59fc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 -- 172.21.15.110:0/4076628753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59fc103340 msgr2=0x7f59fc19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59fc103340 0x7f59fc19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 -- 172.21.15.110:0/4076628753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59fc102980 msgr2=0x7f59fc197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59fc102980 0x7f59fc197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 -- 172.21.15.110:0/4076628753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc19f670 con 0x7f59fc101790 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59fb7fe640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc101790 0x7f59fc197a50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f59f000b450 tx=0x7f59f000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59f8ff9640 1 -- 172.21.15.110:0/4076628753 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0015020 con 0x7f59fc101790 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59f8ff9640 1 -- 172.21.15.110:0/4076628753 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59f0011ce0 con 0x7f59fc101790 2024-04-25T00:01:58.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc19f960 con 0x7f59fc101790 2024-04-25T00:01:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f59f8ff9640 1 -- 172.21.15.110:0/4076628753 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0010dc0 con 0x7f59fc101790 2024-04-25T00:01:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.877+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc19ff30 con 0x7f59fc101790 2024-04-25T00:01:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.878+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc108ab0 con 0x7f59fc101790 2024-04-25T00:01:58.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.879+0000 7f59f8ff9640 1 -- 172.21.15.110:0/4076628753 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f59f0018020 con 0x7f59fc101790 2024-04-25T00:01:58.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.880+0000 7f59f8ff9640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59e40a2b90 0x7f59e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:58.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.880+0000 7f59faffd640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59e40a2b90 0x7f59e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:58.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.880+0000 7f59f8ff9640 1 -- 172.21.15.110:0/4076628753 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f59f00ce890 con 0x7f59fc101790 2024-04-25T00:01:58.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.880+0000 7f59faffd640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59e40a2b90 0x7f59e40a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f59fc198f60 tx=0x7f59ec009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:58.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:58.884+0000 7f59f8ff9640 1 -- 172.21.15.110:0/4076628753 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59f001f020 con 0x7f59fc101790 2024-04-25T00:01:59.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:58 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/901035227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:58 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/162272221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:58 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2573870395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:58 smithi188 ceph-mon[33285]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:59.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:58 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/901035227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:58 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/162272221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:58 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2573870395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:58 smithi110 ceph-mon[29187]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:59.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.062+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59fc062340 con 0x7f59fc101790 2024-04-25T00:01:59.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.063+0000 7f59f8ff9640 1 -- 172.21.15.110:0/4076628753 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7f59f009b820 con 0x7f59fc101790 2024-04-25T00:01:59.064 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:01:59.064 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:01:59.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59e40a2b90 msgr2=0x7f59e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59e40a2b90 0x7f59e40a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f59fc198f60 tx=0x7f59ec009110 comp rx=0 tx=0).stop 2024-04-25T00:01:59.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc101790 msgr2=0x7f59fc197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc101790 0x7f59fc197a50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f59f000b450 tx=0x7f59f000b850 comp rx=0 tx=0).stop 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 shutdown_connections 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59e40a2b90 0x7f59e40a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59fc103340 0x7f59fc19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59fc102980 0x7f59fc197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 --2- 172.21.15.110:0/4076628753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc101790 0x7f59fc197a50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.065+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 >> 172.21.15.110:0/4076628753 conn(0x7f59fc0fcd20 msgr2=0x7f59fc105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.066+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 shutdown_connections 2024-04-25T00:01:59.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.066+0000 7f5a01bed640 1 -- 172.21.15.110:0/4076628753 wait complete. 2024-04-25T00:01:59.077 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:00:02.856564+0000', 'modified': '2024-04-25T00:00:07.553801+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24476, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6835/946560580', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 946560580}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 946560580}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/1627462282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 1627462282}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 1627462282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6837/1170231083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 1170231083}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 1170231083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.188:6839/361433277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 361433277}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 361433277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.173:6837/3367670345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 3367670345}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 3367670345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6839/2519479764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 2519479764}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 2519479764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24457, 24476, 24512]} 2024-04-25T00:01:59.077 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T00:01:59.077 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T00:01:59.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:58 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/901035227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:58 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/162272221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:58 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2573870395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:01:59.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:58 smithi173 ceph-mon[32402]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:01:59.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 -- 172.21.15.110:0/221464035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 msgr2=0x7f6450104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 --2- 172.21.15.110:0/221464035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 0x7f6450104650 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6440009ca0 tx=0x7f644002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 -- 172.21.15.110:0/221464035 shutdown_connections 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 --2- 172.21.15.110:0/221464035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64501074c0 0x7f64501098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 --2- 172.21.15.110:0/221464035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6450104b90 0x7f6450106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 --2- 172.21.15.110:0/221464035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 0x7f6450104650 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 -- 172.21.15.110:0/221464035 >> 172.21.15.110:0/221464035 conn(0x7f64500fbbf0 msgr2=0x7f64500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 -- 172.21.15.110:0/221464035 shutdown_connections 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.175+0000 7f6455ed6640 1 -- 172.21.15.110:0/221464035 wait complete. 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f6455ed6640 1 Processor -- start 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f6455ed6640 1 -- start start 2024-04-25T00:01:59.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f6455ed6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6450102260 0x7f64501979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f6455ed6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450104b90 0x7f6450197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f6455ed6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64501074c0 0x7f645019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f6455ed6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f645010d2a0 con 0x7f6450104b90 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f6455ed6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f645010d120 con 0x7f6450102260 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f6455ed6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f645010d420 con 0x7f64501074c0 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f644effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450104b90 0x7f6450197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f644effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450104b90 0x7f6450197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35326/0 (socket says 172.21.15.110:35326) 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.176+0000 7f644effd640 1 -- 172.21.15.110:0/2023600924 learned_addr learned my addr 172.21.15.110:0/2023600924 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644effd640 1 -- 172.21.15.110:0/2023600924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64501074c0 msgr2=0x7f645019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644effd640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64501074c0 0x7f645019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644effd640 1 -- 172.21.15.110:0/2023600924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6450102260 msgr2=0x7f64501979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644f7fe640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6450102260 0x7f64501979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644effd640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6450102260 0x7f64501979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644effd640 1 -- 172.21.15.110:0/2023600924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645019f5f0 con 0x7f6450104b90 2024-04-25T00:01:59.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644effd640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450104b90 0x7f6450197f10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f644400c970 tx=0x7f644400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644cff9640 1 -- 172.21.15.110:0/2023600924 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444014070 con 0x7f6450104b90 2024-04-25T00:01:59.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644cff9640 1 -- 172.21.15.110:0/2023600924 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6444004440 con 0x7f6450104b90 2024-04-25T00:01:59.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f6455ed6640 1 -- 172.21.15.110:0/2023600924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645019f8e0 con 0x7f6450104b90 2024-04-25T00:01:59.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f644cff9640 1 -- 172.21.15.110:0/2023600924 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444003e50 con 0x7f6450104b90 2024-04-25T00:01:59.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.177+0000 7f6455ed6640 1 -- 172.21.15.110:0/2023600924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645019fe30 con 0x7f6450104b90 2024-04-25T00:01:59.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.179+0000 7f644cff9640 1 -- 172.21.15.110:0/2023600924 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6444017070 con 0x7f6450104b90 2024-04-25T00:01:59.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.179+0000 7f6458be1740 1 Processor -- start 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.179+0000 7f6458be1740 1 -- start start 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6458be1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d3d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6458be1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d42930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6458be1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d45430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6458be1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557d74ca0f00 con 0x557d74d3d440 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6458be1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557d74c49750 con 0x557d74d3de30 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6458be1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557d74c9cec0 con 0x557d74d42fa0 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644cff9640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64380a2d30 0x7f64380a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d3d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d3d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35334/0 (socket says 172.21.15.110:35334) 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 -- 172.21.15.110:0/3300124482 learned_addr learned my addr 172.21.15.110:0/3300124482 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64380a2d30 0x7f64380a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644cff9640 1 -- 172.21.15.110:0/2023600924 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f64440cec40 con 0x7f6450104b90 2024-04-25T00:01:59.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644f7fe640 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d42930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644effd640 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d45430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 -- 172.21.15.110:0/3300124482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 msgr2=0x557d74d45430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d45430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 -- 172.21.15.110:0/3300124482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 msgr2=0x557d74d42930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d42930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 -- 172.21.15.110:0/3300124482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d749b52d0 con 0x557d74d3d440 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644f7fe640 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d42930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d3d860 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6448013a80 tx=0x7f6448013f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c82436466254e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f644ffff640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64380a2d30 0x7f64380a51f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6448010860 tx=0x7f6448015000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6435ffb640 1 -- 172.21.15.110:0/3300124482 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644800ec50 con 0x557d74d3d440 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6435ffb640 1 -- 172.21.15.110:0/3300124482 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6448009970 con 0x557d74d3d440 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.180+0000 7f6435ffb640 1 -- 172.21.15.110:0/3300124482 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448005020 con 0x557d74d3d440 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 -- 172.21.15.110:0/3300124482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 msgr2=0x557d74d3d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d3d860 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6448013a80 tx=0x7f6448013f40 comp rx=0 tx=0).stop 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 -- 172.21.15.110:0/3300124482 shutdown_connections 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d45430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d42930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 --2- 172.21.15.110:0/3300124482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d3d860 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 -- 172.21.15.110:0/3300124482 >> 172.21.15.110:0/3300124482 conn(0x557d74d38ab0 msgr2=0x557d74d38eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 -- 172.21.15.110:0/3300124482 shutdown_connections 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 -- 172.21.15.110:0/3300124482 wait complete. 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.181+0000 7f6458be1740 1 -- start start 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f6458be1740 1 Processor -- start 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f6458be1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f6458be1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d47de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f6458be1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f6458be1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557d74d45ef0 con 0x557d74d3d440 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f6458be1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557d74c5ccf0 con 0x557d74d3de30 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f6458be1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557d74d46070 con 0x557d74d42fa0 2024-04-25T00:01:59.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f6458be1740 10 client.0 _ll_register_callbacks cb 0x557d74c5c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d478a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35344/0 (socket says 172.21.15.110:35344) 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 -- 172.21.15.110:0/1990400861 learned_addr learned my addr 172.21.15.110:0/1990400861 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644ffff640 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644effd640 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d47de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 -- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 msgr2=0x557d74d4ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 -- 172.21.15.110:0/1990400861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 msgr2=0x557d74d47de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d47de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 -- 172.21.15.110:0/1990400861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d74a7f6b0 con 0x557d74d3d440 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644ffff640 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f644f7fe640 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d478a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6440005cc0 tx=0x7f64400093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452599f43aa7aa0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f642effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f642effd640 1 -- 172.21.15.110:0/1990400861 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440031dc0 con 0x557d74d3d440 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.182+0000 7f642effd640 1 -- 172.21.15.110:0/1990400861 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6440003970 con 0x557d74d3d440 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 1 -- 172.21.15.110:0/1990400861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6448009970 con 0x557d74d3d440 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f642effd640 1 -- 172.21.15.110:0/1990400861 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f644004c070 con 0x557d74d3d440 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 10 client.15219 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 1 -- 172.21.15.110:0/1990400861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557d74d51de0 con 0x557d74d3d440 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f642effd640 1 -- 172.21.15.110:0/1990400861 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6448009970 con 0x557d74d3d440 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f642effd640 1 -- 172.21.15.110:0/1990400861 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f64400305c0 con 0x557d74d3d440 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 10 client.15219 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 10 client.15219 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 20 client.15219 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x557d74d54430 0x557d74d558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 4 client.15219 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:59.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f6458be1740 1 -- 172.21.15.110:0/1990400861 --> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d74cae440 con 0x557d74d54430 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.183+0000 7f644effd640 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x557d74d54430 0x557d74d558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.184+0000 7f642effd640 10 client.15219 ms_handle_connect on v2:172.21.15.188:6838/361433277 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.184+0000 7f644effd640 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x557d74d54430 0x557d74d558a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=191e8af5e3f06887 server_cookie=7e5b46293b4b4e5a in_seq=0 out_seq=0 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.184+0000 7f642effd640 1 -- 172.21.15.110:0/1990400861 <== mds.2 v2:172.21.15.188:6838/361433277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d74cae440 con 0x557d74d54430 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.184+0000 7f642effd640 10 client.15219 handle_command_reply: tid=0 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.184+0000 7f6458be1740 1 client.15219 shutdown 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 -- 172.21.15.110:0/1990400861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 msgr2=0x557d74d478a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d478a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6440005cc0 tx=0x7f64400093d0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 -- 172.21.15.110:0/1990400861 shutdown_connections 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x557d74d54430 0x557d74d558a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3de30 0x557d74d47de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 --2- 172.21.15.110:0/1990400861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3d440 0x557d74d478a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f642effd640 0 client.15219 ms_handle_reset on v2:172.21.15.188:6838/361433277 2024-04-25T00:01:59.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 -- 172.21.15.110:0/1990400861 >> 172.21.15.110:0/1990400861 conn(0x557d74d38ab0 msgr2=0x557d74d37ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 -- 172.21.15.110:0/1990400861 shutdown_connections 2024-04-25T00:01:59.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 1 -- 172.21.15.110:0/1990400861 wait complete. 2024-04-25T00:01:59.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.185+0000 7f6458be1740 20 client.15219 trim_cache size 0 max 16384 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.202+0000 7f6458be1740 1 Processor -- start 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f6458be1740 1 -- start start 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f6458be1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d3d440 0x557d74d37250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f6458be1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f6458be1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d42fa0 0x557d74d37790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f6458be1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557d74d53be0 con 0x557d74d3de30 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f6458be1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557d74d542b0 con 0x557d74d42fa0 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f6458be1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557d74d54980 con 0x557d74d3d440 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35348/0 (socket says 172.21.15.110:35348) 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 -- 172.21.15.110:0/4209289402 learned_addr learned my addr 172.21.15.110:0/4209289402 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 -- 172.21.15.110:0/4209289402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d3d440 msgr2=0x557d74d37250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 --2- 172.21.15.110:0/4209289402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d3d440 0x557d74d37250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 -- 172.21.15.110:0/4209289402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d42fa0 msgr2=0x557d74d37790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 --2- 172.21.15.110:0/4209289402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d42fa0 0x557d74d37790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 -- 172.21.15.110:0/4209289402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d74d388e0 con 0x557d74d3de30 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f644effd640 1 --2- 172.21.15.110:0/4209289402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6444097ea0 tx=0x7f6444097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbe83063620b124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f64357fa640 1 -- 172.21.15.110:0/4209289402 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644409b4d0 con 0x557d74d3de30 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f64357fa640 1 -- 172.21.15.110:0/4209289402 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6444080e50 con 0x557d74d3de30 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.203+0000 7f64357fa640 1 -- 172.21.15.110:0/4209289402 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444003680 con 0x557d74d3de30 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 -- 172.21.15.110:0/4209289402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 msgr2=0x557d74d4ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 --2- 172.21.15.110:0/4209289402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6444097ea0 tx=0x7f6444097040 comp rx=0 tx=0).stop 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 -- 172.21.15.110:0/4209289402 shutdown_connections 2024-04-25T00:01:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 --2- 172.21.15.110:0/4209289402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d42fa0 0x557d74d37790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 --2- 172.21.15.110:0/4209289402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4ede0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 --2- 172.21.15.110:0/4209289402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d3d440 0x557d74d37250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 -- 172.21.15.110:0/4209289402 >> 172.21.15.110:0/4209289402 conn(0x557d74d38ab0 msgr2=0x557d74d3f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 -- 172.21.15.110:0/4209289402 shutdown_connections 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 -- 172.21.15.110:0/4209289402 wait complete. 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 -- start start 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 Processor -- start 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.204+0000 7f6458be1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3d440 0x557d74d4af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557d74d55720 con 0x557d74d3de30 2024-04-25T00:01:59.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557d74d55050 con 0x557d74d3d440 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557d74d55df0 con 0x557d74d42fa0 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 10 client.0 _ll_register_callbacks cb 0x7f6440004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35360/0 (socket says 172.21.15.110:35360) 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 -- 172.21.15.110:0/1643911980 learned_addr learned my addr 172.21.15.110:0/1643911980 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644ffff640 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644f7fe640 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3d440 0x557d74d4af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 -- 172.21.15.110:0/1643911980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 msgr2=0x557d74d4e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 -- 172.21.15.110:0/1643911980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3d440 msgr2=0x557d74d4af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3d440 0x557d74d4af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 -- 172.21.15.110:0/1643911980 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d74d4e730 con 0x557d74d3de30 2024-04-25T00:01:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f644effd640 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4b4b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f64440ccae0 tx=0x7f6444003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f86425b50c56988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f642effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f642effd640 1 -- 172.21.15.110:0/1643911980 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64440b3070 con 0x557d74d3de30 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f642effd640 1 -- 172.21.15.110:0/1643911980 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f64440997d0 con 0x557d74d3de30 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 1 -- 172.21.15.110:0/1643911980 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6444080e50 con 0x557d74d3de30 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f642effd640 1 -- 172.21.15.110:0/1643911980 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6444084040 con 0x557d74d3de30 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 10 client.15225 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f6458be1740 1 -- 172.21.15.110:0/1643911980 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557d74dbaac0 con 0x557d74d3de30 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.205+0000 7f642effd640 1 -- 172.21.15.110:0/1643911980 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6444080e50 con 0x557d74d3de30 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f642effd640 1 -- 172.21.15.110:0/1643911980 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f64440a3cb0 con 0x557d74d3de30 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f6458be1740 10 client.15225 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f6458be1740 10 client.15225 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f6458be1740 20 client.15225 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f6458be1740 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x557d74dbf0c0 0x557d74dbddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f6458be1740 4 client.15225 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f6458be1740 1 -- 172.21.15.110:0/1643911980 --> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] -- command(tid 0: {"prefix": "status"}) -- 0x557d74ca8790 con 0x557d74dbf0c0 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f644f7fe640 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x557d74dbf0c0 0x557d74dbddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f644f7fe640 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x557d74dbf0c0 0x557d74dbddd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7313de3a6e77eab2 server_cookie=b771d10744e00c07 in_seq=0 out_seq=0 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f642effd640 10 client.15225 ms_handle_connect on v2:172.21.15.188:6838/361433277 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f642effd640 1 -- 172.21.15.110:0/1643911980 <== mds.2 v2:172.21.15.188:6838/361433277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557d74ca8790 con 0x557d74dbf0c0 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f642effd640 10 client.15225 handle_command_reply: tid=0 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.206+0000 7f6458be1740 1 client.15225 shutdown 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 -- 172.21.15.110:0/1643911980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 msgr2=0x557d74d4b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4b4b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f64440ccae0 tx=0x7f6444003660 comp rx=0 tx=0).stop 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 -- 172.21.15.110:0/1643911980 shutdown_connections 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.188:6838/361433277,v1:172.21.15.188:6839/361433277] conn(0x557d74dbf0c0 0x557d74dbddd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557d74d42fa0 0x557d74d4e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f642effd640 0 client.15225 ms_handle_reset on v2:172.21.15.188:6838/361433277 2024-04-25T00:01:59.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d74d3de30 0x557d74d4b4b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 --2- 172.21.15.110:0/1643911980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557d74d3d440 0x557d74d4af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 -- 172.21.15.110:0/1643911980 >> 172.21.15.110:0/1643911980 conn(0x557d74d38ab0 msgr2=0x557d74d3b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 -- 172.21.15.110:0/1643911980 shutdown_connections 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 1 -- 172.21.15.110:0/1643911980 wait complete. 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6458be1740 20 client.15225 trim_cache size 0 max 16384 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 2, 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 24457, 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-04-25T00:01:59.216 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-04-25T00:01:59.217 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-04-25T00:01:59.217 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 112.657934647, 2024-04-25T00:01:59.217 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 21, 2024-04-25T00:01:59.217 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 87, 2024-04-25T00:01:59.217 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T00:01:59.217 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 130.16275477299999 2024-04-25T00:01:59.217 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:01:59.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 -- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64380a2d30 msgr2=0x7f64380a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64380a2d30 0x7f64380a51f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6448010860 tx=0x7f6448015000 comp rx=0 tx=0).stop 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 -- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450104b90 msgr2=0x7f6450197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450104b90 0x7f6450197f10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f644400c970 tx=0x7f644400ce30 comp rx=0 tx=0).stop 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 -- 172.21.15.110:0/2023600924 shutdown_connections 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64380a2d30 0x7f64380a51f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64501074c0 0x7f645019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450104b90 0x7f6450197f10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 --2- 172.21.15.110:0/2023600924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6450102260 0x7f64501979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 -- 172.21.15.110:0/2023600924 >> 172.21.15.110:0/2023600924 conn(0x7f64500fbbf0 msgr2=0x7f6450102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 -- 172.21.15.110:0/2023600924 shutdown_connections 2024-04-25T00:01:59.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.207+0000 7f6455ed6640 1 -- 172.21.15.110:0/2023600924 wait complete. 2024-04-25T00:01:59.220 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T00:01:59.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 -- 172.21.15.110:0/2113426139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 msgr2=0x7f27dc106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2113426139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 0x7f27dc106f20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009900 tx=0x7f27d002a410 comp rx=0 tx=0).stop 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 -- 172.21.15.110:0/2113426139 shutdown_connections 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2113426139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc107460 0x7f27dc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2113426139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 0x7f27dc106f20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2113426139 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc102200 0x7f27dc1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 -- 172.21.15.110:0/2113426139 >> 172.21.15.110:0/2113426139 conn(0x7f27dc0fbbf0 msgr2=0x7f27dc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 -- 172.21.15.110:0/2113426139 shutdown_connections 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.318+0000 7f27e29a2640 1 -- 172.21.15.110:0/2113426139 wait complete. 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27e29a2640 1 Processor -- start 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27e29a2640 1 -- start start 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27e29a2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc102200 0x7f27dc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27e29a2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 0x7f27dc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27e29a2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc107460 0x7f27dc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27e29a2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc10d2a0 con 0x7f27dc104b30 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27e29a2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc10d120 con 0x7f27dc107460 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27e29a2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc10d420 con 0x7f27dc102200 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 0x7f27dc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 0x7f27dc197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35390/0 (socket says 172.21.15.110:35390) 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.319+0000 7f27db7fe640 1 -- 172.21.15.110:0/2396646667 learned_addr learned my addr 172.21.15.110:0/2396646667 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc102200 0x7f27dc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27db7fe640 1 -- 172.21.15.110:0/2396646667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc102200 msgr2=0x7f27dc197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc102200 0x7f27dc197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27db7fe640 1 -- 172.21.15.110:0/2396646667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc107460 msgr2=0x7f27dc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc107460 0x7f27dc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27db7fe640 1 -- 172.21.15.110:0/2396646667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc19f5f0 con 0x7f27dc104b30 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 0x7f27dc197f60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27d00314e0 tx=0x7f27d00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27d97fa640 1 -- 172.21.15.110:0/2396646667 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d003f030 con 0x7f27dc104b30 2024-04-25T00:01:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27d97fa640 1 -- 172.21.15.110:0/2396646667 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27d0003cb0 con 0x7f27dc104b30 2024-04-25T00:01:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27e29a2640 1 -- 172.21.15.110:0/2396646667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc19f880 con 0x7f27dc104b30 2024-04-25T00:01:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27d97fa640 1 -- 172.21.15.110:0/2396646667 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0035070 con 0x7f27dc104b30 2024-04-25T00:01:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.320+0000 7f27e29a2640 1 -- 172.21.15.110:0/2396646667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc19fd70 con 0x7f27dc104b30 2024-04-25T00:01:59.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.322+0000 7f27d97fa640 1 -- 172.21.15.110:0/2396646667 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f27d0042030 con 0x7f27dc104b30 2024-04-25T00:01:59.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.322+0000 7f27e5791740 1 Processor -- start 2024-04-25T00:01:59.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.322+0000 7f27d97fa640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27c40a2b90 0x7f27c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.322+0000 7f27e5791740 1 -- start start 2024-04-25T00:01:59.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e0f3f640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27c40a2b90 0x7f27c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e5791740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170440 0x55a86a170860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e5791740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a175930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27d97fa640 1 -- 172.21.15.110:0/2396646667 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f27d00ef6e0 con 0x7f27dc104b30 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e5791740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a175fa0 0x55a86a178430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e5791740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a86a0d3f00 con 0x55a86a170e30 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e5791740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a86a07c750 con 0x55a86a175fa0 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e5791740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a86a0cfec0 con 0x55a86a170440 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a175930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a175930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35404/0 (socket says 172.21.15.110:35404) 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 -- 172.21.15.110:0/2451634990 learned_addr learned my addr 172.21.15.110:0/2451634990 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170440 0x55a86a170860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e0f3f640 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a175fa0 0x55a86a178430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 -- 172.21.15.110:0/2451634990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170440 msgr2=0x55a86a170860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170440 0x55a86a170860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 -- 172.21.15.110:0/2451634990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a175fa0 msgr2=0x55a86a178430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a175fa0 0x55a86a178430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 -- 172.21.15.110:0/2451634990 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a869de82d0 con 0x55a86a170e30 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27e0f3f640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27c40a2b90 0x7f27c40a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f27d4005fd0 tx=0x7f27d400c2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a175930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f27d00c46b0 tx=0x7f27d0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4453668f6d981f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27c27fc640 1 -- 172.21.15.110:0/2451634990 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d00d6070 con 0x55a86a170e30 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27c27fc640 1 -- 172.21.15.110:0/2451634990 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27d00c0320 con 0x55a86a170e30 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.323+0000 7f27c27fc640 1 -- 172.21.15.110:0/2451634990 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0049040 con 0x55a86a170e30 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 -- 172.21.15.110:0/2451634990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 msgr2=0x55a86a175930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a175930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f27d00c46b0 tx=0x7f27d0005e80 comp rx=0 tx=0).stop 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 -- 172.21.15.110:0/2451634990 shutdown_connections 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a175fa0 0x55a86a178430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a175930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 --2- 172.21.15.110:0/2451634990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170440 0x55a86a170860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 -- 172.21.15.110:0/2451634990 >> 172.21.15.110:0/2451634990 conn(0x55a86a16bab0 msgr2=0x55a86a16beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 -- 172.21.15.110:0/2451634990 shutdown_connections 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 -- 172.21.15.110:0/2451634990 wait complete. 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 -- start start 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.324+0000 7f27e5791740 1 Processor -- start 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a17a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a17ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a175fa0 0x55a86a181ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a86a178d10 con 0x55a86a170e30 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a86a08fcf0 con 0x55a86a170440 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a86a178e90 con 0x55a86a175fa0 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 10 client.0 _ll_register_callbacks cb 0x55a86a08f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a17ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a17ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35414/0 (socket says 172.21.15.110:35414) 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 -- 172.21.15.110:0/2125869519 learned_addr learned my addr 172.21.15.110:0/2125869519 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a17a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 -- 172.21.15.110:0/2125869519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a175fa0 msgr2=0x55a86a181ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a175fa0 0x55a86a181ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 -- 172.21.15.110:0/2125869519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 msgr2=0x55a86a17a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a17a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 -- 172.21.15.110:0/2125869519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a869eb26b0 con 0x55a86a170e30 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a17ab60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f27d0000c00 tx=0x7f27d0005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8db8263c32fa7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2125869519 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d00d6070 con 0x55a86a170e30 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2125869519 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f27d003b460 con 0x55a86a170e30 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 1 -- 172.21.15.110:0/2125869519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d00c0320 con 0x55a86a170e30 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2125869519 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f27d0049050 con 0x55a86a170e30 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 10 client.15237 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.325+0000 7f27e5791740 1 -- 172.21.15.110:0/2125869519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a86a178d10 con 0x55a86a170e30 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2125869519 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27d00c0320 con 0x55a86a170e30 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2125869519 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f27d00c4750 con 0x55a86a170e30 2024-04-25T00:01:59.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27e5791740 10 client.15237 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27e5791740 10 client.15237 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27e5791740 20 client.15237 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27e5791740 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x55a86a186f60 0x55a86a1883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27e5791740 4 client.15237 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27e5791740 1 -- 172.21.15.110:0/2125869519 --> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86a0e1440 con 0x55a86a186f60 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x55a86a186f60 0x55a86a1883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x55a86a186f60 0x55a86a1883d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c195c09196fe2e9 server_cookie=8baba331342199f9 in_seq=0 out_seq=0 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.326+0000 7f27bb7fe640 10 client.15237 ms_handle_connect on v2:172.21.15.173:6836/3367670345 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2125869519 <== mds.1 v2:172.21.15.173:6836/3367670345 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a86a0e1440 con 0x55a86a186f60 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27bb7fe640 10 client.15237 handle_command_reply: tid=0 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 client.15237 shutdown 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 -- 172.21.15.110:0/2125869519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 msgr2=0x55a86a17ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a17ab60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f27d0000c00 tx=0x7f27d0005b70 comp rx=0 tx=0).stop 2024-04-25T00:01:59.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 -- 172.21.15.110:0/2125869519 shutdown_connections 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x55a86a186f60 0x55a86a1883d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a175fa0 0x55a86a181ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a17ab60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27bb7fe640 0 client.15237 ms_handle_reset on v2:172.21.15.173:6836/3367670345 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 --2- 172.21.15.110:0/2125869519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a17a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.327+0000 7f27e5791740 1 -- 172.21.15.110:0/2125869519 >> 172.21.15.110:0/2125869519 conn(0x55a86a16bab0 msgr2=0x55a86a16aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.328+0000 7f27e5791740 1 -- 172.21.15.110:0/2125869519 shutdown_connections 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.328+0000 7f27e5791740 1 -- 172.21.15.110:0/2125869519 wait complete. 2024-04-25T00:01:59.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.328+0000 7f27e5791740 20 client.15237 trim_cache size 0 max 16384 2024-04-25T00:01:59.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.346+0000 7f27e5791740 1 Processor -- start 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.346+0000 7f27e5791740 1 -- start start 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.346+0000 7f27e5791740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a16bab0 0x55a86a1722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.346+0000 7f27e5791740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a172810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.346+0000 7f27e5791740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170e30 0x55a86a181b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.346+0000 7f27e5791740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a86a186710 con 0x55a86a16bab0 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.346+0000 7f27e5791740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a86a186de0 con 0x55a86a170440 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.346+0000 7f27e5791740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a86a1874b0 con 0x55a86a170e30 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a16bab0 0x55a86a1722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a16bab0 0x55a86a1722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35422/0 (socket says 172.21.15.110:35422) 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 -- 172.21.15.110:0/2727563664 learned_addr learned my addr 172.21.15.110:0/2727563664 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27e0f3f640 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170e30 0x55a86a181b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a172810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 -- 172.21.15.110:0/2727563664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170e30 msgr2=0x55a86a181b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170e30 0x55a86a181b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 -- 172.21.15.110:0/2727563664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 msgr2=0x55a86a172810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a172810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 -- 172.21.15.110:0/2727563664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86a16b8e0 con 0x55a86a16bab0 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a16bab0 0x55a86a1722d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55a86a171690 tx=0x7f27cc00d580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aee44a45601fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27c1ffb640 1 -- 172.21.15.110:0/2727563664 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc01bdc0 con 0x55a86a16bab0 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27c1ffb640 1 -- 172.21.15.110:0/2727563664 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27cc020070 con 0x55a86a16bab0 2024-04-25T00:01:59.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.347+0000 7f27c1ffb640 1 -- 172.21.15.110:0/2727563664 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc003850 con 0x55a86a16bab0 2024-04-25T00:01:59.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 -- 172.21.15.110:0/2727563664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a16bab0 msgr2=0x55a86a1722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a16bab0 0x55a86a1722d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55a86a171690 tx=0x7f27cc00d580 comp rx=0 tx=0).stop 2024-04-25T00:01:59.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 -- 172.21.15.110:0/2727563664 shutdown_connections 2024-04-25T00:01:59.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a170e30 0x55a86a181b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a172810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 --2- 172.21.15.110:0/2727563664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a16bab0 0x55a86a1722d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 -- 172.21.15.110:0/2727563664 >> 172.21.15.110:0/2727563664 conn(0x55a86a1794f0 msgr2=0x55a86a17a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 -- 172.21.15.110:0/2727563664 shutdown_connections 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 -- 172.21.15.110:0/2727563664 wait complete. 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.348+0000 7f27e5791740 1 -- start start 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e5791740 1 Processor -- start 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e5791740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a16bab0 0x55a86a17dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e5791740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a17e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e5791740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a1811f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e5791740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a86a188250 con 0x55a86a170e30 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e5791740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a86a187b80 con 0x55a86a170440 2024-04-25T00:01:59.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e5791740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a86a188920 con 0x55a86a16bab0 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e5791740 10 client.0 _ll_register_callbacks cb 0x7f27d00bb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e0f3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a1811f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e0f3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a1811f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35436/0 (socket says 172.21.15.110:35436) 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e0f3f640 1 -- 172.21.15.110:0/2264521938 learned_addr learned my addr 172.21.15.110:0/2264521938 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a16bab0 0x55a86a17dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27db7fe640 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a17e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.349+0000 7f27e0f3f640 1 -- 172.21.15.110:0/2264521938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a16bab0 msgr2=0x55a86a17dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27e0f3f640 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a16bab0 0x55a86a17dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27e0f3f640 1 -- 172.21.15.110:0/2264521938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 msgr2=0x55a86a17e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27e0f3f640 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a17e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27e0f3f640 1 -- 172.21.15.110:0/2264521938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86a1817e0 con 0x55a86a170e30 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27e0f3f640 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a1811f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f27d4003f60 tx=0x7f27d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b8a4e003c216ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2264521938 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4022020 con 0x55a86a170e30 2024-04-25T00:01:59.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2264521938 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f27d40119c0 con 0x55a86a170e30 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27e5791740 1 -- 172.21.15.110:0/2264521938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27cc020070 con 0x55a86a170e30 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2264521938 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f27d4011c00 con 0x55a86a170e30 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27e5791740 10 client.15243 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27e5791740 1 -- 172.21.15.110:0/2264521938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a86a1eda20 con 0x55a86a170e30 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2264521938 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27cc020070 con 0x55a86a170e30 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.350+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2264521938 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f27d4029400 con 0x55a86a170e30 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27e5791740 10 client.15243 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27e5791740 10 client.15243 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27e5791740 20 client.15243 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27e5791740 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x55a86a175fa0 0x55a86a1f1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27e5791740 4 client.15243 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27e5791740 1 -- 172.21.15.110:0/2264521938 --> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] -- command(tid 0: {"prefix": "status"}) -- 0x55a86a17a410 con 0x55a86a175fa0 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x55a86a175fa0 0x55a86a1f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27dbfff640 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x55a86a175fa0 0x55a86a1f1460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=875f31e8d810bd1b server_cookie=fa3543e015251189 in_seq=0 out_seq=0 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27bb7fe640 10 client.15243 ms_handle_connect on v2:172.21.15.173:6836/3367670345 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27bb7fe640 1 -- 172.21.15.110:0/2264521938 <== mds.1 v2:172.21.15.173:6836/3367670345 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a86a17a410 con 0x55a86a175fa0 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27bb7fe640 10 client.15243 handle_command_reply: tid=0 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.351+0000 7f27e5791740 1 client.15243 shutdown 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 -- 172.21.15.110:0/2264521938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 msgr2=0x55a86a1811f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a1811f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f27d4003f60 tx=0x7f27d4004a40 comp rx=0 tx=0).stop 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 -- 172.21.15.110:0/2264521938 shutdown_connections 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.173:6836/3367670345,v1:172.21.15.173:6837/3367670345] conn(0x55a86a175fa0 0x55a86a1f1460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a86a170e30 0x55a86a1811f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a86a170440 0x55a86a17e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27bb7fe640 0 client.15243 ms_handle_reset on v2:172.21.15.173:6836/3367670345 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 --2- 172.21.15.110:0/2264521938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a86a16bab0 0x55a86a17dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 -- 172.21.15.110:0/2264521938 >> 172.21.15.110:0/2264521938 conn(0x55a86a1794f0 msgr2=0x55a86a17a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 -- 172.21.15.110:0/2264521938 shutdown_connections 2024-04-25T00:01:59.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 1 -- 172.21.15.110:0/2264521938 wait complete. 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e5791740 20 client.15243 trim_cache size 0 max 16384 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 1, 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 24476, 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 113.80795822100001, 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 21, 2024-04-25T00:01:59.358 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 87, 2024-04-25T00:01:59.359 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T00:01:59.359 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 133.12486794099999 2024-04-25T00:01:59.359 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:01:59.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e29a2640 1 -- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27c40a2b90 msgr2=0x7f27c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27c40a2b90 0x7f27c40a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f27d4005fd0 tx=0x7f27d400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e29a2640 1 -- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 msgr2=0x7f27dc197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.352+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 0x7f27dc197f60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27d00314e0 tx=0x7f27d00042e0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.353+0000 7f27e29a2640 1 -- 172.21.15.110:0/2396646667 shutdown_connections 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.353+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27c40a2b90 0x7f27c40a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.353+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc107460 0x7f27dc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.353+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27dc104b30 0x7f27dc197f60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.353+0000 7f27e29a2640 1 --2- 172.21.15.110:0/2396646667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc102200 0x7f27dc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.353+0000 7f27e29a2640 1 -- 172.21.15.110:0/2396646667 >> 172.21.15.110:0/2396646667 conn(0x7f27dc0fbbf0 msgr2=0x7f27dc102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.353+0000 7f27e29a2640 1 -- 172.21.15.110:0/2396646667 shutdown_connections 2024-04-25T00:01:59.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.353+0000 7f27e29a2640 1 -- 172.21.15.110:0/2396646667 wait complete. 2024-04-25T00:01:59.371 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:01:59.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 -- 172.21.15.110:0/3975379529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37580ffa50 msgr2=0x7f37580ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 --2- 172.21.15.110:0/3975379529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37580ffa50 0x7f37580ffe70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f374c009900 tx=0x7f374c02a410 comp rx=0 tx=0).stop 2024-04-25T00:01:59.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 -- 172.21.15.110:0/3975379529 shutdown_connections 2024-04-25T00:01:59.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 --2- 172.21.15.110:0/3975379529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3758101170 0x7f3758105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 --2- 172.21.15.110:0/3975379529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37581007b0 0x7f3758100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 --2- 172.21.15.110:0/3975379529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37580ffa50 0x7f37580ffe70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 -- 172.21.15.110:0/3975379529 >> 172.21.15.110:0/3975379529 conn(0x7f37580fb380 msgr2=0x7f37580fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 -- 172.21.15.110:0/3975379529 shutdown_connections 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 -- 172.21.15.110:0/3975379529 wait complete. 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 Processor -- start 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.471+0000 7f375f1ce640 1 -- start start 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375f1ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37580ffa50 0x7f375819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375f1ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37581007b0 0x7f375819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375f1ce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3758101170 0x7f37581a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375f1ce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37581138f0 con 0x7f37581007b0 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375f1ce640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3758113770 con 0x7f3758101170 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375f1ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3758113a70 con 0x7f37580ffa50 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37581007b0 0x7f375819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37581007b0 0x7f375819e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35462/0 (socket says 172.21.15.110:35462) 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 -- 172.21.15.110:0/1745349854 learned_addr learned my addr 172.21.15.110:0/1745349854 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375d76b640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3758101170 0x7f37581a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375cf6a640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37580ffa50 0x7f375819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 -- 172.21.15.110:0/1745349854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37580ffa50 msgr2=0x7f375819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37580ffa50 0x7f375819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 -- 172.21.15.110:0/1745349854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3758101170 msgr2=0x7f37581a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3758101170 0x7f37581a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 -- 172.21.15.110:0/1745349854 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37581a5ba0 con 0x7f37581007b0 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375cf6a640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37580ffa50 0x7f375819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3757fff640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37581007b0 0x7f375819e4c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f374400c970 tx=0x7f374400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3755ffb640 1 -- 172.21.15.110:0/1745349854 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744004220 con 0x7f37581007b0 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3755ffb640 1 -- 172.21.15.110:0/1745349854 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37440043c0 con 0x7f37581007b0 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f3755ffb640 1 -- 172.21.15.110:0/1745349854 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374400f450 con 0x7f37581007b0 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.472+0000 7f375f1ce640 1 -- 172.21.15.110:0/1745349854 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37581a5e90 con 0x7f37581007b0 2024-04-25T00:01:59.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.473+0000 7f375f1ce640 1 -- 172.21.15.110:0/1745349854 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37581a6350 con 0x7f37581007b0 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.475+0000 7f3755ffb640 1 -- 172.21.15.110:0/1745349854 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f37440079b0 con 0x7f37581007b0 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.475+0000 7f3761fb0740 1 Processor -- start 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.475+0000 7f3761fb0740 1 -- start start 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.475+0000 7f3755ffb640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37480a2b90 0x7f37480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x560223920860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375d76b640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37480a2b90 0x7f37480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 0x560223925930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223925fa0 0x560223928430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3761fb0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560223883f00 con 0x560223920440 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3761fb0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56022382c750 con 0x560223920e30 2024-04-25T00:01:59.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3761fb0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56022387fec0 con 0x560223925fa0 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3755ffb640 1 -- 172.21.15.110:0/1745349854 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f37440cee50 con 0x7f37581007b0 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x560223920860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x560223920860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35476/0 (socket says 172.21.15.110:35476) 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 -- 172.21.15.110:0/3401750279 learned_addr learned my addr 172.21.15.110:0/3401750279 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3757fff640 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 0x560223925930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375d76b640 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223925fa0 0x560223928430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 -- 172.21.15.110:0/3401750279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223925fa0 msgr2=0x560223928430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223925fa0 0x560223928430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 -- 172.21.15.110:0/3401750279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 msgr2=0x560223925930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 0x560223925930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 -- 172.21.15.110:0/3401750279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602235982d0 con 0x560223920440 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375d76b640 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223925fa0 0x560223928430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:01:59.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375d76b640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37480a2b90 0x7f37480a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3750007e60 tx=0x7f375000f790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f375cf6a640 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x560223920860 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f374c009900 tx=0x7f374c031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c9a1da181ee6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3742ffd640 1 -- 172.21.15.110:0/3401750279 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c035030 con 0x560223920440 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3742ffd640 1 -- 172.21.15.110:0/3401750279 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c031e20 con 0x560223920440 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.476+0000 7f3742ffd640 1 -- 172.21.15.110:0/3401750279 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c003c50 con 0x560223920440 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 -- 172.21.15.110:0/3401750279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 msgr2=0x560223920860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x560223920860 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f374c009900 tx=0x7f374c031b70 comp rx=0 tx=0).stop 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 -- 172.21.15.110:0/3401750279 shutdown_connections 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223925fa0 0x560223928430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 0x560223925930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 --2- 172.21.15.110:0/3401750279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x560223920860 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 -- 172.21.15.110:0/3401750279 >> 172.21.15.110:0/3401750279 conn(0x56022391bab0 msgr2=0x56022391beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 -- 172.21.15.110:0/3401750279 shutdown_connections 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 -- 172.21.15.110:0/3401750279 wait complete. 2024-04-25T00:01:59.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.477+0000 7f3761fb0740 1 -- start start 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3761fb0740 1 Processor -- start 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920440 0x56022392a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 0x56022392acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223925fa0 0x560223931cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3761fb0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560223928ef0 con 0x560223925fa0 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3761fb0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56022383fcf0 con 0x560223920e30 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3761fb0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560223929070 con 0x560223920440 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3761fb0740 10 client.0 _ll_register_callbacks cb 0x56022383f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223925fa0 0x560223931cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223925fa0 0x560223931cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35492/0 (socket says 172.21.15.110:35492) 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 -- 172.21.15.110:0/1172518770 learned_addr learned my addr 172.21.15.110:0/1172518770 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3757fff640 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 0x56022392acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375cf6a640 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920440 0x56022392a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 -- 172.21.15.110:0/1172518770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920440 msgr2=0x56022392a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920440 0x56022392a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 -- 172.21.15.110:0/1172518770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 msgr2=0x56022392acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 0x56022392acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 -- 172.21.15.110:0/1172518770 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602236626b0 con 0x560223925fa0 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f375d76b640 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223925fa0 0x560223931cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3750007930 tx=0x7f3750004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de07e17836ab4a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.478+0000 7f3737fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3737fff640 1 -- 172.21.15.110:0/1172518770 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750022070 con 0x560223925fa0 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3737fff640 1 -- 172.21.15.110:0/1172518770 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f375001e440 con 0x560223925fa0 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 1 -- 172.21.15.110:0/1172518770 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c031e20 con 0x560223925fa0 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3737fff640 1 -- 172.21.15.110:0/1172518770 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f375001e640 con 0x560223925fa0 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 10 client.15255 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 1 -- 172.21.15.110:0/1172518770 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560223934ce0 con 0x560223925fa0 2024-04-25T00:01:59.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3737fff640 1 -- 172.21.15.110:0/1172518770 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c031e20 con 0x560223925fa0 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3737fff640 1 -- 172.21.15.110:0/1172518770 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3750026760 con 0x560223925fa0 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 10 client.15255 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 20 client.15255 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560223937430 0x5602239388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 4 client.15255 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f3761fb0740 1 -- 172.21.15.110:0/1172518770 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560223891440 con 0x560223937430 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.479+0000 7f375cf6a640 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560223937430 0x5602239388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.480+0000 7f3737fff640 10 client.15255 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.480+0000 7f375cf6a640 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560223937430 0x5602239388a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cee11433fa69d7ba server_cookie=31b690aecad015e1 in_seq=0 out_seq=0 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3737fff640 1 -- 172.21.15.110:0/1172518770 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560223891440 con 0x560223937430 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3737fff640 10 client.15255 handle_command_reply: tid=0 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 client.15255 shutdown 2024-04-25T00:01:59.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 -- 172.21.15.110:0/1172518770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223925fa0 msgr2=0x560223931cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223925fa0 0x560223931cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3750007930 tx=0x7f3750004ab0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 -- 172.21.15.110:0/1172518770 shutdown_connections 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560223937430 0x5602239388a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223925fa0 0x560223931cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3737fff640 0 client.15255 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223920e30 0x56022392acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1172518770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920440 0x56022392a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 -- 172.21.15.110:0/1172518770 >> 172.21.15.110:0/1172518770 conn(0x56022391bab0 msgr2=0x56022391aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 -- 172.21.15.110:0/1172518770 shutdown_connections 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 1 -- 172.21.15.110:0/1172518770 wait complete. 2024-04-25T00:01:59.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.481+0000 7f3761fb0740 20 client.15255 trim_cache size 0 max 16384 2024-04-25T00:01:59.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.498+0000 7f3761fb0740 1 Processor -- start 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f3761fb0740 1 -- start start 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x5602239221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920e30 0x560223931c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223925fa0 0x560223922700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f3761fb0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560223936be0 con 0x560223920440 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f3761fb0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602239372b0 con 0x560223925fa0 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f3761fb0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560223937980 con 0x560223920e30 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x5602239221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x5602239221c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35500/0 (socket says 172.21.15.110:35500) 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 -- 172.21.15.110:0/1243698542 learned_addr learned my addr 172.21.15.110:0/1243698542 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 -- 172.21.15.110:0/1243698542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920e30 msgr2=0x560223931c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 --2- 172.21.15.110:0/1243698542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920e30 0x560223931c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 -- 172.21.15.110:0/1243698542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223925fa0 msgr2=0x560223922700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 --2- 172.21.15.110:0/1243698542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223925fa0 0x560223922700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 -- 172.21.15.110:0/1243698542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560223921210 con 0x560223920440 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f375cf6a640 1 --2- 172.21.15.110:0/1243698542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x5602239221c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f374c006010 tx=0x7f374c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb2a7172127d102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f37427fc640 1 -- 172.21.15.110:0/1243698542 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c055070 con 0x560223920440 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f37427fc640 1 -- 172.21.15.110:0/1243698542 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c050860 con 0x560223920440 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.499+0000 7f37427fc640 1 -- 172.21.15.110:0/1243698542 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c035070 con 0x560223920440 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 -- 172.21.15.110:0/1243698542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 msgr2=0x5602239221c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1243698542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x5602239221c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f374c006010 tx=0x7f374c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 -- 172.21.15.110:0/1243698542 shutdown_connections 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1243698542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223925fa0 0x560223922700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1243698542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920e30 0x560223931c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 --2- 172.21.15.110:0/1243698542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x5602239221c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 -- 172.21.15.110:0/1243698542 >> 172.21.15.110:0/1243698542 conn(0x56022391bab0 msgr2=0x560223916690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 -- 172.21.15.110:0/1243698542 shutdown_connections 2024-04-25T00:01:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 -- 172.21.15.110:0/1243698542 wait complete. 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 -- start start 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.500+0000 7f3761fb0740 1 Processor -- start 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x56022392d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920e30 0x56022392ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223925fa0 0x560223930ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560223938720 con 0x560223920440 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560223938050 con 0x560223925fa0 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560223938df0 con 0x560223920e30 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 10 client.0 _ll_register_callbacks cb 0x7f3750010880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x56022392d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x56022392d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35506/0 (socket says 172.21.15.110:35506) 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 -- 172.21.15.110:0/64837729 learned_addr learned my addr 172.21.15.110:0/64837729 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 -- 172.21.15.110:0/64837729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920e30 msgr2=0x56022392ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920e30 0x56022392ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 -- 172.21.15.110:0/64837729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223925fa0 msgr2=0x560223930ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223925fa0 0x560223930ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 -- 172.21.15.110:0/64837729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602239314e0 con 0x560223920440 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f375cf6a640 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x56022392d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f374c009420 tx=0x7f374c050a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6233973c494f1851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3737fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3737fff640 1 -- 172.21.15.110:0/64837729 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c055040 con 0x560223920440 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3737fff640 1 -- 172.21.15.110:0/64837729 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f374c004240 con 0x560223920440 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 1 -- 172.21.15.110:0/64837729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c050860 con 0x560223920440 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3737fff640 1 -- 172.21.15.110:0/64837729 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f374c035040 con 0x560223920440 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 10 client.15261 fetch_fsmap learned FSMap version 21 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3761fb0740 1 -- 172.21.15.110:0/64837729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56022399da00 con 0x560223920440 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.501+0000 7f3737fff640 1 -- 172.21.15.110:0/64837729 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c050860 con 0x560223920440 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3737fff640 1 -- 172.21.15.110:0/64837729 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f374c061030 con 0x560223920440 2024-04-25T00:01:59.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3761fb0740 10 client.15261 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3761fb0740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3761fb0740 20 client.15261 populate_metadata read hostname 'smithi110' 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3761fb0740 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5602239a1e90 0x5602239a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3761fb0740 4 client.15261 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3761fb0740 1 -- 172.21.15.110:0/64837729 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "status"}) -- 0x56022388b790 con 0x5602239a1e90 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3757fff640 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5602239a1e90 0x5602239a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3737fff640 10 client.15261 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.502+0000 7f3757fff640 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5602239a1e90 0x5602239a0ba0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e05df50ef0e474 server_cookie=a05727d3f2956f4b in_seq=0 out_seq=0 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3737fff640 1 -- 172.21.15.110:0/64837729 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56022388b790 con 0x5602239a1e90 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3737fff640 10 client.15261 handle_command_reply: tid=0 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 client.15261 shutdown 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 -- 172.21.15.110:0/64837729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 msgr2=0x56022392d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x56022392d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f374c009420 tx=0x7f374c050a10 comp rx=0 tx=0).stop 2024-04-25T00:01:59.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 -- 172.21.15.110:0/64837729 shutdown_connections 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5602239a1e90 0x5602239a0ba0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560223925fa0 0x560223930ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560223920e30 0x56022392ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3737fff640 0 client.15261 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 --2- 172.21.15.110:0/64837729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560223920440 0x56022392d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 -- 172.21.15.110:0/64837729 >> 172.21.15.110:0/64837729 conn(0x56022391bab0 msgr2=0x560223916690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 -- 172.21.15.110:0/64837729 shutdown_connections 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 1 -- 172.21.15.110:0/64837729 wait complete. 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f3761fb0740 20 client.15261 trim_cache size 0 max 16384 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:01:59.509 INFO:teuthology.orchestra.run.smithi110.stdout: "cluster_fsid": "787fa75a-0295-11ef-bc93-c7b262605968", 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "whoami": 0, 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 24512, 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "want_state": "up:active", 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "state": "up:active", 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs", 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "rank_uptime": 115.983543944, 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "mdsmap_epoch": 21, 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch": 87, 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout: "uptime": 121.68719388700001 2024-04-25T00:01:59.510 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:01:59.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 -- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37480a2b90 msgr2=0x7f37480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37480a2b90 0x7f37480a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3750007e60 tx=0x7f375000f790 comp rx=0 tx=0).stop 2024-04-25T00:01:59.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 -- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37581007b0 msgr2=0x7f375819e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:01:59.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37581007b0 0x7f375819e4c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f374400c970 tx=0x7f374400ce30 comp rx=0 tx=0).stop 2024-04-25T00:01:59.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 -- 172.21.15.110:0/1745349854 shutdown_connections 2024-04-25T00:01:59.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37480a2b90 0x7f37480a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3758101170 0x7f37581a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37581007b0 0x7f375819e4c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 --2- 172.21.15.110:0/1745349854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37580ffa50 0x7f375819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:01:59.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 -- 172.21.15.110:0/1745349854 >> 172.21.15.110:0/1745349854 conn(0x7f37580fb380 msgr2=0x7f37581019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:01:59.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.503+0000 7f375f1ce640 1 -- 172.21.15.110:0/1745349854 shutdown_connections 2024-04-25T00:01:59.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:01:59.504+0000 7f375f1ce640 1 -- 172.21.15.110:0/1745349854 wait complete. 2024-04-25T00:01:59.514 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:01:59.535 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:01:59.535 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:01:59.535 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:01:59.535 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T00:01:59.558 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T00:01:59.559 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-25T00:01:59.559 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-04-25T00:01:59.559 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:01:59.559 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-04-25T00:01:59.559 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-04-25T00:01:59.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:59.559 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-04-25T00:01:59.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:01:59.575 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-04-25T00:01:59.640 INFO:teuthology.orchestra.run.smithi110.stderr:Cannot find device "ceph-brx" 2024-04-25T00:01:59.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:01:59.642 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T00:01:59.642 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T00:01:59.931 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:02:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:59 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/888007332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:02:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:01:59 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4076628753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:02:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:59 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/888007332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:02:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:01:59 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4076628753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:02:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:59 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/888007332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:02:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:01:59 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4076628753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:02:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:00 smithi188 ceph-mon[33285]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:02:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:00 smithi110 ceph-mon[29187]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:02:01.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:00 smithi173 ceph-mon[32402]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:02:01.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 -- 172.21.15.110:0/3732573049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be806a8e0 msgr2=0x7f1be810f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 --2- 172.21.15.110:0/3732573049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be806a8e0 0x7f1be810f0f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b0a0 tx=0x7f1be400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 -- 172.21.15.110:0/3732573049 shutdown_connections 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 --2- 172.21.15.110:0/3732573049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be806a8e0 0x7f1be810f0f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 --2- 172.21.15.110:0/3732573049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be81082b0 0x7f1be8108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 --2- 172.21.15.110:0/3732573049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1be8069f40 0x7f1be806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 -- 172.21.15.110:0/3732573049 >> 172.21.15.110:0/3732573049 conn(0x7f1be80780e0 msgr2=0x7f1be80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 -- 172.21.15.110:0/3732573049 shutdown_connections 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.809+0000 7f1befa31700 1 -- 172.21.15.110:0/3732573049 wait complete. 2024-04-25T00:02:01.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1befa31700 1 Processor -- start 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1befa31700 1 -- start start 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1befa31700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1be8069f40 0x7f1be8101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1befa31700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be806a8e0 0x7f1be8101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1befa31700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be81082b0 0x7f1be8101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1befa31700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81123e0 con 0x7f1be81082b0 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1befa31700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1be8112260 con 0x7f1be806a8e0 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1befa31700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1be8112560 con 0x7f1be8069f40 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1bedfce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be81082b0 0x7f1be8101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1bedfce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be81082b0 0x7f1be8101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35528/0 (socket says 172.21.15.110:35528) 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1bedfce700 1 -- 172.21.15.110:0/544230355 learned_addr learned my addr 172.21.15.110:0/544230355 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1becfcc700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be806a8e0 0x7f1be8101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:01.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1bedfce700 1 -- 172.21.15.110:0/544230355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1be8069f40 msgr2=0x7f1be8101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1bedfce700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1be8069f40 0x7f1be8101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1bedfce700 1 -- 172.21.15.110:0/544230355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be806a8e0 msgr2=0x7f1be8101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1bedfce700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be806a8e0 0x7f1be8101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1bedfce700 1 -- 172.21.15.110:0/544230355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be81a6e80 con 0x7f1be81082b0 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.810+0000 7f1becfcc700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be806a8e0 0x7f1be8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.811+0000 7f1bedfce700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be81082b0 0x7f1be8101c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b780 tx=0x7f1be4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.811+0000 7f1bde7fc700 1 -- 172.21.15.110:0/544230355 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4021070 con 0x7f1be81082b0 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.811+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be81a7110 con 0x7f1be81082b0 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.811+0000 7f1bde7fc700 1 -- 172.21.15.110:0/544230355 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1be4007e30 con 0x7f1be81082b0 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.811+0000 7f1bde7fc700 1 -- 172.21.15.110:0/544230355 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4004800 con 0x7f1be81082b0 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.811+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be81a7570 con 0x7f1be81082b0 2024-04-25T00:02:01.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.812+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be8068f20 con 0x7f1be81082b0 2024-04-25T00:02:01.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.813+0000 7f1bde7fc700 1 -- 172.21.15.110:0/544230355 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1be4007980 con 0x7f1be81082b0 2024-04-25T00:02:01.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.813+0000 7f1bde7fc700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1bd40a2c20 0x7f1bd40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:01.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.813+0000 7f1bed7cd700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1bd40a2c20 0x7f1bd40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:01.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.813+0000 7f1bde7fc700 1 -- 172.21.15.110:0/544230355 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1be40d0000 con 0x7f1be81082b0 2024-04-25T00:02:01.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.814+0000 7f1bed7cd700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1bd40a2c20 0x7f1bd40a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009b60 tx=0x7f1be0009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:01.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:01.817+0000 7f1bde7fc700 1 -- 172.21.15.110:0/544230355 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1be409d020 con 0x7f1be81082b0 2024-04-25T00:02:02.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.058+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1be802dcf0 con 0x7f1be81082b0 2024-04-25T00:02:02.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.058+0000 7f1bde7fc700 1 -- 172.21.15.110:0/544230355 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7f1be409c920 con 0x7f1be81082b0 2024-04-25T00:02:02.059 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:02:02.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1bd40a2c20 msgr2=0x7f1bd40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1bd40a2c20 0x7f1bd40a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009b60 tx=0x7f1be0009620 comp rx=0 tx=0).stop 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be81082b0 msgr2=0x7f1be8101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be81082b0 0x7f1be8101c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b780 tx=0x7f1be4004d00 comp rx=0 tx=0).stop 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 shutdown_connections 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1bd40a2c20 0x7f1bd40a50e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1be81082b0 0x7f1be8101c80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1be8069f40 0x7f1be8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 --2- 172.21.15.110:0/544230355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be806a8e0 0x7f1be8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 >> 172.21.15.110:0/544230355 conn(0x7f1be80780e0 msgr2=0x7f1be80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 shutdown_connections 2024-04-25T00:02:02.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:02.060+0000 7f1befa31700 1 -- 172.21.15.110:0/544230355 wait complete. 2024-04-25T00:02:02.351 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:02 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/544230355' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:02:02.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:02 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/544230355' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:02:02.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:02 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/544230355' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:02:02.897 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:02:02.898 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:02:02.898 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-04-25T00:02:02.898 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:02:02.898 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-04-25T00:02:02.898 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:02:02.898 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:02:02.898 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:02:02.898 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-04-25T00:02:02.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:02.898 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-04-25T00:02:02.914 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:02:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:02:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:02:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-04-25T00:02:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42146sec preferred_lft 42146sec 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::ff83:2cf2:308:5b44/64 scope link noprefixroute 2024-04-25T00:02:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:02:02.916 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:02:02.916 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:02:02.916 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T00:02:02.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:02:02.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip addr flush dev ceph-brx 2024-04-25T00:02:02.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set ceph-brx up 2024-04-25T00:02:02.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:02:02.916 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T00:02:03.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:02:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:03.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:03.110 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:02:03.124 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:03 smithi110 ceph-mon[29187]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:02:03.135 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-04-25T00:02:03.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:03.136 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-04-25T00:02:03.202 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-04-25T00:02:03.202 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:02:03.202 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:02:03.203 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:02:03.203 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:02:03.203 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:02:03.203 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T00:02:03.203 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:02:03.204 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:02:03.204 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:02:03.204 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T00:02:03.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:02:03.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:03.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:03.482 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-04-25T00:02:03.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:03.498 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list-id 2024-04-25T00:02:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:03 smithi188 ceph-mon[33285]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:02:03.553 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:02:03.553 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T00:02:03.553 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns add ceph-ns--tmp-cephfs 2024-04-25T00:02:03.553 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-04-25T00:02:03.553 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T00:02:03.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:03 smithi173 ceph-mon[32402]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:02:03.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:02:03.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:03.669 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-04-25T00:02:03.670 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:02:03.670 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T00:02:03.670 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-04-25T00:02:03.670 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:02:03.670 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-04-25T00:02:03.671 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-04-25T00:02:03.671 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-04-25T00:02:03.671 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T00:02:03.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:02:03.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:03.790 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:02:03.790 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T00:02:03.790 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-04-25T00:02:03.790 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:02:03.790 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T00:02:03.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:02:03.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:03.864 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-04-25T00:02:03.864 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:02:03.864 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /tmp/cephfs 2024-04-25T00:02:03.883 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/tmp/cephfs' 2024-04-25T00:02:03.883 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:02:03.883 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /tmp/cephfs 2024-04-25T00:02:03.937 DEBUG:teuthology.orchestra.run.smithi110:> sudo modprobe fuse 2024-04-25T00:02:04.002 DEBUG:teuthology.orchestra.run.smithi110:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:02:04.059 INFO:teuthology.orchestra.run.smithi110.stdout:/proc 2024-04-25T00:02:04.059 INFO:teuthology.orchestra.run.smithi110.stdout:/sys 2024-04-25T00:02:04.059 INFO:teuthology.orchestra.run.smithi110.stdout:/dev 2024-04-25T00:02:04.060 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/security 2024-04-25T00:02:04.060 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/shm 2024-04-25T00:02:04.060 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/pts 2024-04-25T00:02:04.060 INFO:teuthology.orchestra.run.smithi110.stdout:/run 2024-04-25T00:02:04.061 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/cgroup 2024-04-25T00:02:04.061 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/pstore 2024-04-25T00:02:04.061 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/bpf 2024-04-25T00:02:04.061 INFO:teuthology.orchestra.run.smithi110.stdout:/ 2024-04-25T00:02:04.061 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/selinux 2024-04-25T00:02:04.061 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:02:04.061 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/tracing 2024-04-25T00:02:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/mqueue 2024-04-25T00:02:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/debug 2024-04-25T00:02:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/hugepages 2024-04-25T00:02:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/config 2024-04-25T00:02:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:02:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/fuse/connections 2024-04-25T00:02:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:02:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:02:04.063 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:02:04.063 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:02:04.063 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/1000 2024-04-25T00:02:04.063 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/ceph 2024-04-25T00:02:04.063 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:02:04.063 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/a6f53561dfd97c5cc5333c1d686ff58a215a4651bd4f8dd8aa47d3e116c6435b/merged 2024-04-25T00:02:04.064 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/ec83a2fddc40673783cc69e327a41c4ec2c48d9352c3e0d6980628f0d03fc937/merged 2024-04-25T00:02:04.064 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/0 2024-04-25T00:02:04.064 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:02:04.064 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/ab4d48c244adba9df649886c45804aa9821c9f483509d5dc9cca6144f652539e/merged 2024-04-25T00:02:04.064 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/dfd188210aa4a0fd93c3516c8bae3745b1b56479aa52c0ed84db98b233e64f06/merged 2024-04-25T00:02:04.064 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/bd7a4daf58f626d1177685bb410a7db8bbd2dd857d76b308884ba3d66848697a/merged 2024-04-25T00:02:04.064 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/b1fef891024de2a7ef9d16f86e90d90b0ba95f1203f45d918cb08314e8114f7e/merged 2024-04-25T00:02:04.065 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/f71338afd71182050f004d47627b9f8522c52f539b43782abd30dcdbe743f3a8/merged 2024-04-25T00:02:04.065 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/fcec989be7ec5d3dd1caad61f3fe2e9ea3d3d1a78ba6d7020439c78e99fff396/merged 2024-04-25T00:02:04.065 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/7e779775c01df699f06c89853c7d4c4ca4565d592c526782809349653818e76a/merged 2024-04-25T00:02:04.065 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/21090a36d833d3972e01539bb718cbd80cf7aea1febd39f1565d74e000e75c6b/merged 2024-04-25T00:02:04.065 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns 2024-04-25T00:02:04.065 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-25T00:02:04.066 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-25T00:02:04.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:04.066 DEBUG:teuthology.orchestra.run.smithi110:> ls /sys/fs/fuse/connections 2024-04-25T00:02:04.114 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:02:04.114 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/a6bd4ddc-e93b-4dad-bcd9-0ea4116814e4 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-04-25T00:02:04.156 DEBUG:teuthology.orchestra.run.smithi110:> sudo modprobe fuse 2024-04-25T00:02:04.179 DEBUG:teuthology.orchestra.run.smithi110:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:02:04.234 INFO:teuthology.orchestra.run.smithi110.stdout:/proc 2024-04-25T00:02:04.234 INFO:teuthology.orchestra.run.smithi110.stdout:/sys 2024-04-25T00:02:04.234 INFO:teuthology.orchestra.run.smithi110.stdout:/dev 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/security 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/shm 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/pts 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/run 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/cgroup 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/pstore 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/bpf 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/ 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/selinux 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/tracing 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/mqueue 2024-04-25T00:02:04.235 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/debug 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/hugepages 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/config 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/fuse/connections 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/1000 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/ceph 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:02:04.236 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/a6f53561dfd97c5cc5333c1d686ff58a215a4651bd4f8dd8aa47d3e116c6435b/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/ec83a2fddc40673783cc69e327a41c4ec2c48d9352c3e0d6980628f0d03fc937/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/0 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/ab4d48c244adba9df649886c45804aa9821c9f483509d5dc9cca6144f652539e/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/dfd188210aa4a0fd93c3516c8bae3745b1b56479aa52c0ed84db98b233e64f06/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/bd7a4daf58f626d1177685bb410a7db8bbd2dd857d76b308884ba3d66848697a/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/b1fef891024de2a7ef9d16f86e90d90b0ba95f1203f45d918cb08314e8114f7e/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/f71338afd71182050f004d47627b9f8522c52f539b43782abd30dcdbe743f3a8/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/fcec989be7ec5d3dd1caad61f3fe2e9ea3d3d1a78ba6d7020439c78e99fff396/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/7e779775c01df699f06c89853c7d4c4ca4565d592c526782809349653818e76a/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/21090a36d833d3972e01539bb718cbd80cf7aea1febd39f1565d74e000e75c6b/merged 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-25T00:02:04.237 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-25T00:02:04.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:04.238 DEBUG:teuthology.orchestra.run.smithi110:> ls /sys/fs/fuse/connections 2024-04-25T00:02:04.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi110.stderr:2024-04-25T00:02:04.248+0000 7ff34b823480 -1 init, newargv = 0x564f219154c0 newargc=17 2024-04-25T00:02:04.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi110.stderr:ceph-fuse[80833]: starting ceph client 2024-04-25T00:02:04.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi110.stderr:ceph-fuse[80833]: starting fuse 2024-04-25T00:02:05.365 DEBUG:teuthology.orchestra.run.smithi110:> sudo modprobe fuse 2024-04-25T00:02:05.396 DEBUG:teuthology.orchestra.run.smithi110:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:02:05.454 INFO:teuthology.orchestra.run.smithi110.stdout:/proc 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/sys 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/dev 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/security 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/shm 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/pts 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/run 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/cgroup 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/pstore 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/bpf 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/ 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/selinux 2024-04-25T00:02:05.455 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/tracing 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/mqueue 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/debug 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/hugepages 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/config 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/fuse/connections 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:02:05.456 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:02:05.457 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:02:05.457 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/1000 2024-04-25T00:02:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/ceph 2024-04-25T00:02:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:02:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/a6f53561dfd97c5cc5333c1d686ff58a215a4651bd4f8dd8aa47d3e116c6435b/merged 2024-04-25T00:02:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/ec83a2fddc40673783cc69e327a41c4ec2c48d9352c3e0d6980628f0d03fc937/merged 2024-04-25T00:02:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/0 2024-04-25T00:02:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:02:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/ab4d48c244adba9df649886c45804aa9821c9f483509d5dc9cca6144f652539e/merged 2024-04-25T00:02:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/dfd188210aa4a0fd93c3516c8bae3745b1b56479aa52c0ed84db98b233e64f06/merged 2024-04-25T00:02:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/bd7a4daf58f626d1177685bb410a7db8bbd2dd857d76b308884ba3d66848697a/merged 2024-04-25T00:02:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/b1fef891024de2a7ef9d16f86e90d90b0ba95f1203f45d918cb08314e8114f7e/merged 2024-04-25T00:02:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/f71338afd71182050f004d47627b9f8522c52f539b43782abd30dcdbe743f3a8/merged 2024-04-25T00:02:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/fcec989be7ec5d3dd1caad61f3fe2e9ea3d3d1a78ba6d7020439c78e99fff396/merged 2024-04-25T00:02:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/7e779775c01df699f06c89853c7d4c4ca4565d592c526782809349653818e76a/merged 2024-04-25T00:02:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/21090a36d833d3972e01539bb718cbd80cf7aea1febd39f1565d74e000e75c6b/merged 2024-04-25T00:02:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns 2024-04-25T00:02:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-25T00:02:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-25T00:02:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs 2024-04-25T00:02:05.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:05.460 DEBUG:teuthology.orchestra.run.smithi110:> ls /sys/fs/fuse/connections 2024-04-25T00:02:05.511 INFO:teuthology.orchestra.run.smithi110.stdout:115 2024-04-25T00:02:05.512 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> import glob 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> import re 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> import os 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> import subprocess 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> def _find_admin_socket(client_name): 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> files = glob.glob(asok_path) 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> mountpoint = "/tmp/cephfs" 2024-04-25T00:02:05.512 DEBUG:teuthology.orchestra.run.smithi110:> 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> # Given a non-glob path, it better be there 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> if "*" not in asok_path: 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> assert(len(files) == 1) 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> return files[0] 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> for f in files: 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> contents = proc_f.read() 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> if mountpoint in contents: 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> return f 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> print(_find_admin_socket("client.0")) 2024-04-25T00:02:05.513 DEBUG:teuthology.orchestra.run.smithi110:> ' 2024-04-25T00:02:05.635 INFO:teuthology.orchestra.run.smithi110.stdout:/var/run/ceph/ceph-client.0.80833.asok 2024-04-25T00:02:05.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:02:05.645 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:05 smithi110 ceph-mon[29187]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:02:05.646 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80833.asok 2024-04-25T00:02:05.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:05.646 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80833.asok status 2024-04-25T00:02:05.780 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:02:05.780 INFO:teuthology.orchestra.run.smithi110.stdout: "metadata": { 2024-04-25T00:02:05.780 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:02:05.780 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:02:05.780 INFO:teuthology.orchestra.run.smithi110.stdout: "entity_id": "0", 2024-04-25T00:02:05.780 INFO:teuthology.orchestra.run.smithi110.stdout: "hostname": "smithi110", 2024-04-25T00:02:05.780 INFO:teuthology.orchestra.run.smithi110.stdout: "mount_point": "/tmp/cephfs", 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "pid": "80833", 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "root": "/volumes/qa/sv_0/a6bd4ddc-e93b-4dad-bcd9-0ea4116814e4" 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "dentry_count": 0, 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "dentry_pinned_count": 0, 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 15273, 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "inst": { 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "name": { 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "client", 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "num": 15273 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": { 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v1", 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:02:05.781 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 1904814552 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": { 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v1", 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 1904814552 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "inst_str": "client.15273 192.168.0.1:0/1904814552", 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "addr_str": "192.168.0.1:0/1904814552", 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "inode_count": 5, 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_epoch": 21, 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_epoch": 87, 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:02:05.782 INFO:teuthology.orchestra.run.smithi110.stdout: "blocklisted": false, 2024-04-25T00:02:05.783 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs" 2024-04-25T00:02:05.783 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:02:05.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:05.789 DEBUG:teuthology.orchestra.run.smithi110:> stat --file-system '--printf=%T 2024-04-25T00:02:05.789 DEBUG:teuthology.orchestra.run.smithi110:> ' -- /tmp/cephfs 2024-04-25T00:02:05.803 INFO:teuthology.orchestra.run.smithi110.stdout:fuseblk 2024-04-25T00:02:05.804 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-04-25T00:02:05.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:02:05.804 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /tmp/cephfs 2024-04-25T00:02:05.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:05 smithi173 ceph-mon[32402]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:02:05.869 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T00:02:05.879 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:02:05.879 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T00:02:05.879 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T00:02:06.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 -- 172.21.15.110:0/850243686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4101940 msgr2=0x7fcfe4108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/850243686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4101940 0x7fcfe4108260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009ca0 tx=0x7fcfdc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 -- 172.21.15.110:0/850243686 shutdown_connections 2024-04-25T00:02:06.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/850243686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4101940 0x7fcfe4108260 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/850243686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfe4100f80 0x7fcfe4101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/850243686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe40ffe30 0x7fcfe4100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 -- 172.21.15.110:0/850243686 >> 172.21.15.110:0/850243686 conn(0x7fcfe40fb3a0 msgr2=0x7fcfe40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:06.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 -- 172.21.15.110:0/850243686 shutdown_connections 2024-04-25T00:02:06.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.008+0000 7fcfea6ee640 1 -- 172.21.15.110:0/850243686 wait complete. 2024-04-25T00:02:06.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfea6ee640 1 Processor -- start 2024-04-25T00:02:06.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfea6ee640 1 -- start start 2024-04-25T00:02:06.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfea6ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe40ffe30 0x7fcfe40fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfea6ee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4100f80 0x7fcfe419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfea6ee640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfe4101940 0x7fcfe419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfea6ee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe410b9e0 con 0x7fcfe4100f80 2024-04-25T00:02:06.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfea6ee640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe410b860 con 0x7fcfe4101940 2024-04-25T00:02:06.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfea6ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe410bb60 con 0x7fcfe40ffe30 2024-04-25T00:02:06.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4100f80 0x7fcfe419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4100f80 0x7fcfe419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35536/0 (socket says 172.21.15.110:35536) 2024-04-25T00:02:06.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe37fe640 1 -- 172.21.15.110:0/1082062815 learned_addr learned my addr 172.21.15.110:0/1082062815 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:06.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe8c8b640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfe4101940 0x7fcfe419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe37fe640 1 -- 172.21.15.110:0/1082062815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe40ffe30 msgr2=0x7fcfe40fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe3fff640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe40ffe30 0x7fcfe40fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe37fe640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe40ffe30 0x7fcfe40fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe37fe640 1 -- 172.21.15.110:0/1082062815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfe4101940 msgr2=0x7fcfe419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe37fe640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfe4101940 0x7fcfe419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe37fe640 1 -- 172.21.15.110:0/1082062815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe419f560 con 0x7fcfe4100f80 2024-04-25T00:02:06.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe8c8b640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfe4101940 0x7fcfe419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:06.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.009+0000 7fcfe3fff640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe40ffe30 0x7fcfe40fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:06.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.010+0000 7fcfe37fe640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4100f80 0x7fcfe419c630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800c970 tx=0x7fcfd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.010+0000 7fcfe17fa640 1 -- 172.21.15.110:0/1082062815 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8007c10 con 0x7fcfe4100f80 2024-04-25T00:02:06.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.010+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe419f7f0 con 0x7fcfe4100f80 2024-04-25T00:02:06.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.010+0000 7fcfe17fa640 1 -- 172.21.15.110:0/1082062815 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfd8007db0 con 0x7fcfe4100f80 2024-04-25T00:02:06.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.010+0000 7fcfe17fa640 1 -- 172.21.15.110:0/1082062815 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800f7d0 con 0x7fcfe4100f80 2024-04-25T00:02:06.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.010+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe419fd40 con 0x7fcfe4100f80 2024-04-25T00:02:06.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.011+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe4107430 con 0x7fcfe4100f80 2024-04-25T00:02:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:05 smithi188 ceph-mon[33285]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:02:06.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.012+0000 7fcfe17fa640 1 -- 172.21.15.110:0/1082062815 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcfd8018480 con 0x7fcfe4100f80 2024-04-25T00:02:06.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.012+0000 7fcfe17fa640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcfcc0a2c30 0x7fcfcc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:06.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.012+0000 7fcfe3fff640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcfcc0a2c30 0x7fcfcc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:06.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.012+0000 7fcfe17fa640 1 -- 172.21.15.110:0/1082062815 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcfd80cf410 con 0x7fcfe4100f80 2024-04-25T00:02:06.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.013+0000 7fcfe3fff640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcfcc0a2c30 0x7fcfcc0a50f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4006fd0 tx=0x7fcfd4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:06.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.017+0000 7fcfe17fa640 1 -- 172.21.15.110:0/1082062815 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcfd809c430 con 0x7fcfe4100f80 2024-04-25T00:02:06.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.229+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fcfe4062340 con 0x7fcfe4100f80 2024-04-25T00:02:06.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.598+0000 7fcfe17fa640 1 -- 172.21.15.110:0/1082062815 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v22) ==== 117+0+0 (secure 0 0 0) 0x7fcfd809bd30 con 0x7fcfe4100f80 2024-04-25T00:02:06.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcfcc0a2c30 msgr2=0x7fcfcc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcfcc0a2c30 0x7fcfcc0a50f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4006fd0 tx=0x7fcfd4008040 comp rx=0 tx=0).stop 2024-04-25T00:02:06.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4100f80 msgr2=0x7fcfe419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:06.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4100f80 0x7fcfe419c630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800c970 tx=0x7fcfd800ce30 comp rx=0 tx=0).stop 2024-04-25T00:02:06.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 shutdown_connections 2024-04-25T00:02:06.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fcfcc0a2c30 0x7fcfcc0a50f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfe4101940 0x7fcfe419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe4100f80 0x7fcfe419c630 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 --2- 172.21.15.110:0/1082062815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfe40ffe30 0x7fcfe40fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 >> 172.21.15.110:0/1082062815 conn(0x7fcfe40fb3a0 msgr2=0x7fcfe4104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 shutdown_connections 2024-04-25T00:02:06.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:02:06.600+0000 7fcfea6ee640 1 -- 172.21.15.110:0/1082062815 wait complete. 2024-04-25T00:02:06.612 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T00:02:06.744 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T00:02:06.756 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T00:02:06.756 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T00:02:06.759 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T00:02:06.770 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:02:06.771 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T00:02:06.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-04-25T00:02:06.804 INFO:teuthology.orchestra.run.smithi110.stdout:total 0 2024-04-25T00:02:06.815 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem Size Used Avail Use% Mounted on 2024-04-25T00:02:06.815 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-fuse 340G 0 340G 0% /tmp/cephfs 2024-04-25T00:02:06.817 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-04-25T00:02:06.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:06 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1082062815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:02:06.877 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:06 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1082062815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:02:06.907 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-04-25T00:02:06.984 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-04-25T00:02:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:06 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1082062815' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:02:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:07 smithi173 ceph-mon[32402]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:02:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:07 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1082062815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:02:07.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:07 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:07 smithi188 ceph-mon[33285]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:02:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:07 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1082062815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:02:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:07 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:07 smithi110 ceph-mon[29187]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:02:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:07 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1082062815' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:02:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:07 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:08.434 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-04-25T00:02:08.468 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-04-25T00:02:08.470 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-04-25T00:02:08.548 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-04-25T00:02:08.626 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/: 2024-04-25T00:02:08.626 INFO:teuthology.orchestra.run.smithi110.stdout:total 2 2024-04-25T00:02:08.626 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:08.626 INFO:teuthology.orchestra.run.smithi110.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 .. 2024-04-25T00:02:08.626 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 data 2024-04-25T00:02:08.627 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:08.627 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data: 2024-04-25T00:02:08.627 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:08.627 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:08.627 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 .. 2024-04-25T00:02:08.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-04-25T00:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout:[Service] 2024-04-25T00:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-04-25T00:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-04-25T00:02:08.711 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-04-25T00:02:08.829 INFO:teuthology.orchestra.run.smithi110.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-04-25T00:02:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:09 smithi188 ceph-mon[33285]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-25T00:02:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:09 smithi110 ceph-mon[29187]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-25T00:02:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:09 smithi173 ceph-mon[32402]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-25T00:02:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:11 smithi188 ceph-mon[33285]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T00:02:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:11 smithi110 ceph-mon[29187]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T00:02:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:11 smithi173 ceph-mon[32402]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T00:02:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:13 smithi188 ceph-mon[33285]: pgmap v356: 97 pgs: 97 active+clean; 3.0 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 246 KiB/s wr, 19 op/s 2024-04-25T00:02:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:13 smithi110 ceph-mon[29187]: pgmap v356: 97 pgs: 97 active+clean; 3.0 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 246 KiB/s wr, 19 op/s 2024-04-25T00:02:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:13 smithi173 ceph-mon[32402]: pgmap v356: 97 pgs: 97 active+clean; 3.0 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 246 KiB/s wr, 19 op/s 2024-04-25T00:02:16.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:15 smithi188 ceph-mon[33285]: pgmap v357: 97 pgs: 97 active+clean; 24 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.3 MiB/s wr, 49 op/s 2024-04-25T00:02:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:15 smithi110 ceph-mon[29187]: pgmap v357: 97 pgs: 97 active+clean; 24 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.3 MiB/s wr, 49 op/s 2024-04-25T00:02:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:15 smithi173 ceph-mon[32402]: pgmap v357: 97 pgs: 97 active+clean; 24 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.3 MiB/s wr, 49 op/s 2024-04-25T00:02:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:17 smithi188 ceph-mon[33285]: pgmap v358: 97 pgs: 97 active+clean; 28 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-04-25T00:02:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:17 smithi110 ceph-mon[29187]: pgmap v358: 97 pgs: 97 active+clean; 28 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-04-25T00:02:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:17 smithi173 ceph-mon[32402]: pgmap v358: 97 pgs: 97 active+clean; 28 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-04-25T00:02:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.896+0000 7f5fe654c640 1 -- 172.21.15.110:0/4086688087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe0102250 msgr2=0x7f5fe0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.896+0000 7f5fe654c640 1 --2- 172.21.15.110:0/4086688087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe0102250 0x7f5fe0108ae0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009ca0 tx=0x7f5fd802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 -- 172.21.15.110:0/4086688087 shutdown_connections 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 --2- 172.21.15.110:0/4086688087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe0102250 0x7f5fe0108ae0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 --2- 172.21.15.110:0/4086688087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0101890 0x7f5fe0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 --2- 172.21.15.110:0/4086688087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fe01006a0 0x7f5fe0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 -- 172.21.15.110:0/4086688087 >> 172.21.15.110:0/4086688087 conn(0x7f5fe00fbbf0 msgr2=0x7f5fe00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 -- 172.21.15.110:0/4086688087 shutdown_connections 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 -- 172.21.15.110:0/4086688087 wait complete. 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 Processor -- start 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.897+0000 7f5fe654c640 1 -- start start 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fe654c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe01006a0 0x7f5fe0104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fe654c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0101890 0x7f5fe0102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fe654c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fe0102250 0x7f5fe01031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fe654c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe010c1d0 con 0x7f5fe01006a0 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fe654c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe010c050 con 0x7f5fe0102250 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fe654c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe010c350 con 0x7f5fe0101890 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe01006a0 0x7f5fe0104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe01006a0 0x7f5fe0104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35638/0 (socket says 172.21.15.110:35638) 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 -- 172.21.15.110:0/856849693 learned_addr learned my addr 172.21.15.110:0/856849693 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0101890 0x7f5fe0102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fe4ae9640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fe0102250 0x7f5fe01031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 -- 172.21.15.110:0/856849693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0101890 msgr2=0x7f5fe0102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0101890 0x7f5fe0102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 -- 172.21.15.110:0/856849693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fe0102250 msgr2=0x7f5fe01031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fe0102250 0x7f5fe01031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 -- 172.21.15.110:0/856849693 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe01036f0 con 0x7f5fe01006a0 2024-04-25T00:02:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdffff640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe01006a0 0x7f5fe0104630 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000e9c0 tx=0x7f5fd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdd7fa640 1 -- 172.21.15.110:0/856849693 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd000cd80 con 0x7f5fe01006a0 2024-04-25T00:02:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fe654c640 1 -- 172.21.15.110:0/856849693 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe01039e0 con 0x7f5fe01006a0 2024-04-25T00:02:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdd7fa640 1 -- 172.21.15.110:0/856849693 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fd00035f0 con 0x7f5fe01006a0 2024-04-25T00:02:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.898+0000 7f5fdd7fa640 1 -- 172.21.15.110:0/856849693 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0010690 con 0x7f5fe01006a0 2024-04-25T00:02:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.899+0000 7f5fe654c640 1 -- 172.21.15.110:0/856849693 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe01a6410 con 0x7f5fe01006a0 2024-04-25T00:02:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fdd7fa640 1 -- 172.21.15.110:0/856849693 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5fd00037a0 con 0x7f5fe01006a0 2024-04-25T00:02:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fe9258740 1 Processor -- start 2024-04-25T00:02:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fe9258740 1 -- start start 2024-04-25T00:02:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8cce50 0x55c67b8cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8d29b0 0x55c67b8d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fe9258740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c67b830cb0 con 0x55c67b8cd840 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fe9258740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c67b7d95b0 con 0x55c67b8d29b0 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.901+0000 7f5fe9258740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c67b82cc70 con 0x55c67b8cce50 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdd7fa640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fc80a2c30 0x7f5fc80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdf7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8d29b0 0x55c67b8d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35646/0 (socket says 172.21.15.110:35646) 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdffff640 1 -- 172.21.15.110:0/2163402083 learned_addr learned my addr 172.21.15.110:0/2163402083 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fe4ae9640 1 --2- 172.21.15.110:0/2163402083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8cce50 0x55c67b8cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdd7fa640 1 -- 172.21.15.110:0/856849693 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5fd00cf380 con 0x7f5fe01006a0 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdf7fe640 1 -- 172.21.15.110:0/2163402083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8cce50 msgr2=0x55c67b8cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/2163402083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8cce50 0x55c67b8cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdf7fe640 1 -- 172.21.15.110:0/2163402083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 msgr2=0x55c67b8d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/2163402083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdf7fe640 1 -- 172.21.15.110:0/2163402083 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67b545210 con 0x55c67b8d29b0 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fe4ae9640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fc80a2c30 0x7f5fc80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/2163402083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8d29b0 0x55c67b8d4e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009bd0 tx=0x7f5fd4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=342651c5aeec6c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.902+0000 7f5fc67fc640 1 -- 172.21.15.110:0/2163402083 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4004230 con 0x55c67b8d29b0 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fc67fc640 1 -- 172.21.15.110:0/2163402083 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fd4009e40 con 0x55c67b8d29b0 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fc67fc640 1 -- 172.21.15.110:0/2163402083 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4004d30 con 0x55c67b8d29b0 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 -- 172.21.15.110:0/2163402083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8d29b0 msgr2=0x55c67b8d4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2163402083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8d29b0 0x55c67b8d4e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009bd0 tx=0x7f5fd4011810 comp rx=0 tx=0).stop 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe4ae9640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fc80a2c30 0x7f5fc80a50f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8004af0 tx=0x7f5fd802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 -- 172.21.15.110:0/2163402083 shutdown_connections 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2163402083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8d29b0 0x55c67b8d4e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2163402083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2163402083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8cce50 0x55c67b8cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 -- 172.21.15.110:0/2163402083 >> 172.21.15.110:0/2163402083 conn(0x55c67b8c84c0 msgr2=0x55c67b8c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 -- 172.21.15.110:0/2163402083 shutdown_connections 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.903+0000 7f5fe9258740 1 -- 172.21.15.110:0/2163402083 wait complete. 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 1 -- start start 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 1 Processor -- start 2024-04-25T00:02:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b8de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c67b8d5960 con 0x55c67b8cd840 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c67b7ecb50 con 0x55c67b8cce50 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c67b8d5ae0 con 0x55c67b8d29b0 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8d7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:45070/0 (socket says 172.21.15.110:45070) 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fdffff640 1 -- 172.21.15.110:0/3378723450 learned_addr learned my addr 172.21.15.110:0/3378723450 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe9258740 10 client.0 _ll_register_callbacks cb 0x55c67b827ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fe4ae9640 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b8de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.904+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fdffff640 1 -- 172.21.15.110:0/3378723450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 msgr2=0x55c67b8de7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fdffff640 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b8de7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fdffff640 1 -- 172.21.15.110:0/3378723450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 msgr2=0x55c67b8d77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fdffff640 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fdffff640 1 -- 172.21.15.110:0/3378723450 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67b611620 con 0x55c67b8cce50 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fdffff640 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8d7270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00a4a40 tx=0x7f5fd00a4880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc85e7ab846afff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/3378723450 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd009c070 con 0x55c67b8cce50 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fe9258740 1 -- 172.21.15.110:0/3378723450 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c67b83e1f0 con 0x55c67b8cce50 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/3378723450 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5fd009b060 con 0x55c67b8cce50 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/3378723450 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5fd009bc10 con 0x55c67b8cce50 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fe9258740 10 client.24872 fetch_fsmap learned FSMap version 22 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fe9258740 1 -- 172.21.15.110:0/3378723450 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c67b8e1c20 con 0x55c67b8cce50 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.905+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/3378723450 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c67b83e1f0 con 0x55c67b8cce50 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/3378723450 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5fd009ab60 con 0x55c67b8cce50 2024-04-25T00:02:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fe9258740 10 client.24872 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T00:02:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fe9258740 10 client.24872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:02:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fe9258740 20 client.24872 populate_metadata read hostname 'smithi110' 2024-04-25T00:02:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c67b8e3240 0x55c67b8e5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fe9258740 4 client.24872 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:02:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fe9258740 1 -- 172.21.15.110:0/3378723450 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c67b838540 con 0x55c67b8e3240 2024-04-25T00:02:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c67b8e3240 0x55c67b8e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c67b8e3240 0x55c67b8e5690 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba518e15976f078 server_cookie=a8a306932b7fd9a9 in_seq=0 out_seq=0 2024-04-25T00:02:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.906+0000 7f5fbf7fe640 10 client.24872 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/3378723450 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c67b838540 con 0x55c67b8e3240 2024-04-25T00:02:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fbf7fe640 10 client.24872 handle_command_reply: tid=0 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 client.24872 shutdown 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 -- 172.21.15.110:0/3378723450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 msgr2=0x55c67b8d7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8d7270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00a4a40 tx=0x7f5fd00a4880 comp rx=0 tx=0).stop 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 -- 172.21.15.110:0/3378723450 shutdown_connections 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c67b8e3240 0x55c67b8e5690 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b8de7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fbf7fe640 0 client.24872 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8d77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3378723450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8d7270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 -- 172.21.15.110:0/3378723450 >> 172.21.15.110:0/3378723450 conn(0x55c67b8c84c0 msgr2=0x55c67b8c75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 -- 172.21.15.110:0/3378723450 shutdown_connections 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 1 -- 172.21.15.110:0/3378723450 wait complete. 2024-04-25T00:02:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.908+0000 7f5fe9258740 20 client.24872 trim_cache size 0 max 16384 2024-04-25T00:02:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.924+0000 7f5fe9258740 1 Processor -- start 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.924+0000 7f5fe9258740 1 -- start start 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.924+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cce50 0x55c67b8d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.924+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cd840 0x55c67b8de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.924+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b8d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.924+0000 7f5fe9258740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01b1940 con 0x55c67b8cce50 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.924+0000 7f5fe9258740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01af210 con 0x55c67b8cd840 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.924+0000 7f5fe9258740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01ad060 con 0x55c67b8d29b0 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cce50 0x55c67b8d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cce50 0x55c67b8d6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35670/0 (socket says 172.21.15.110:35670) 2024-04-25T00:02:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 -- 172.21.15.110:0/3245990914 learned_addr learned my addr 172.21.15.110:0/3245990914 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cd840 0x55c67b8de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fe4ae9640 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b8d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 -- 172.21.15.110:0/3245990914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 msgr2=0x55c67b8d73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b8d73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 -- 172.21.15.110:0/3245990914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cd840 msgr2=0x55c67b8de7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cd840 0x55c67b8de7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 -- 172.21.15.110:0/3245990914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67b8c8060 con 0x55c67b8cce50 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fdffff640 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cce50 0x55c67b8d6e80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00cd220 tx=0x7f5fd009a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f4eefa0519b8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fc5ffb640 1 -- 172.21.15.110:0/3245990914 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd007e070 con 0x55c67b8cce50 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fc5ffb640 1 -- 172.21.15.110:0/3245990914 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fd0003b30 con 0x55c67b8cce50 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.925+0000 7f5fc5ffb640 1 -- 172.21.15.110:0/3245990914 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00a14f0 con 0x55c67b8cce50 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 -- 172.21.15.110:0/3245990914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cce50 msgr2=0x55c67b8d6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cce50 0x55c67b8d6e80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00cd220 tx=0x7f5fd009a120 comp rx=0 tx=0).stop 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 -- 172.21.15.110:0/3245990914 shutdown_connections 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b8d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cd840 0x55c67b8de7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 --2- 172.21.15.110:0/3245990914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cce50 0x55c67b8d6e80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 -- 172.21.15.110:0/3245990914 >> 172.21.15.110:0/3245990914 conn(0x55c67b8c84c0 msgr2=0x55c67b8cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 -- 172.21.15.110:0/3245990914 shutdown_connections 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 -- 172.21.15.110:0/3245990914 wait complete. 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.926+0000 7f5fe9258740 1 -- start start 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe9258740 1 Processor -- start 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8db7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe9258740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b947800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe9258740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01ad6d0 con 0x55c67b8cd840 2024-04-25T00:02:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe9258740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01aeb40 con 0x55c67b8cce50 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe9258740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01adda0 con 0x55c67b8d29b0 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe9258740 10 client.0 _ll_register_callbacks cb 0x7f5fd00ccdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8db7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8db7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:45080/0 (socket says 172.21.15.110:45080) 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8dbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35676/0 (socket says 172.21.15.110:35676) 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdffff640 1 -- 172.21.15.110:0/2417815633 learned_addr learned my addr 172.21.15.110:0/2417815633 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fe4ae9640 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b947800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdffff640 1 -- 172.21.15.110:0/2417815633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 msgr2=0x55c67b947800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdffff640 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b947800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdffff640 1 -- 172.21.15.110:0/2417815633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 msgr2=0x55c67b8dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdffff640 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8dbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdffff640 1 -- 172.21.15.110:0/2417815633 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67b947d40 con 0x55c67b8cce50 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.927+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fdffff640 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8db7d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5fd009bdd0 tx=0x7f5fd00a49b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f16c1e8020adce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/2417815633 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd007e070 con 0x55c67b8cce50 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/2417815633 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5fd009b060 con 0x55c67b8cce50 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 1 -- 172.21.15.110:0/2417815633 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c67b7b68e0 con 0x55c67b8cce50 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/2417815633 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5fd009ffc0 con 0x55c67b8cce50 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 10 client.24878 fetch_fsmap learned FSMap version 22 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 1 -- 172.21.15.110:0/2417815633 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c67b947d40 con 0x55c67b8cce50 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/2417815633 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c67b7b68e0 con 0x55c67b8cce50 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/2417815633 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5fd0081030 con 0x55c67b8cce50 2024-04-25T00:02:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 10 client.24878 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T00:02:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 10 client.24878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:02:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 20 client.24878 populate_metadata read hostname 'smithi110' 2024-04-25T00:02:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c67b9494a0 0x55c67b94b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 4 client.24878 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:02:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.928+0000 7f5fe9258740 1 -- 172.21.15.110:0/2417815633 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c67b802540 con 0x55c67b9494a0 2024-04-25T00:02:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.929+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c67b9494a0 0x55c67b94b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.929+0000 7f5fdf7fe640 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c67b9494a0 0x55c67b94b890 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca02caa05281d1f3 server_cookie=aa17484194b72318 in_seq=0 out_seq=0 2024-04-25T00:02:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:19.929+0000 7f5fbf7fe640 10 client.24878 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:19 smithi188 ceph-mon[33285]: pgmap v359: 97 pgs: 97 active+clean; 37 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.1 MiB/s wr, 97 op/s 2024-04-25T00:02:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:19 smithi110 ceph-mon[29187]: pgmap v359: 97 pgs: 97 active+clean; 37 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.1 MiB/s wr, 97 op/s 2024-04-25T00:02:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:19 smithi173 ceph-mon[32402]: pgmap v359: 97 pgs: 97 active+clean; 37 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.1 MiB/s wr, 97 op/s 2024-04-25T00:02:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fbf7fe640 1 -- 172.21.15.110:0/2417815633 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c67b802540 con 0x55c67b9494a0 2024-04-25T00:02:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fbf7fe640 10 client.24878 handle_command_reply: tid=0 2024-04-25T00:02:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 client.24878 shutdown 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 -- 172.21.15.110:0/2417815633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 msgr2=0x55c67b8db7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8db7d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5fd009bdd0 tx=0x7f5fd00a49b0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 -- 172.21.15.110:0/2417815633 shutdown_connections 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c67b9494a0 0x55c67b94b890 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67b8d29b0 0x55c67b947800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c67b8cd840 0x55c67b8dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 --2- 172.21.15.110:0/2417815633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c67b8cce50 0x55c67b8db7d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fbf7fe640 0 client.24878 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 -- 172.21.15.110:0/2417815633 >> 172.21.15.110:0/2417815633 conn(0x55c67b8c84c0 msgr2=0x55c67b8d6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.406+0000 7f5fe9258740 1 -- 172.21.15.110:0/2417815633 shutdown_connections 2024-04-25T00:02:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe9258740 1 -- 172.21.15.110:0/2417815633 wait complete. 2024-04-25T00:02:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe9258740 20 client.24878 trim_cache size 0 max 16384 2024-04-25T00:02:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:02:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:02:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:02:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 -- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fc80a2c30 msgr2=0x7f5fc80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fc80a2c30 0x7f5fc80a50f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8004af0 tx=0x7f5fd802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 -- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe01006a0 msgr2=0x7f5fe0104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe01006a0 0x7f5fe0104630 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000e9c0 tx=0x7f5fd000ee80 comp rx=0 tx=0).stop 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 -- 172.21.15.110:0/856849693 shutdown_connections 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fc80a2c30 0x7f5fc80a50f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fe0102250 0x7f5fe01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe0101890 0x7f5fe0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 --2- 172.21.15.110:0/856849693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fe01006a0 0x7f5fe0104630 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 -- 172.21.15.110:0/856849693 >> 172.21.15.110:0/856849693 conn(0x7f5fe00fbbf0 msgr2=0x7f5fe00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 -- 172.21.15.110:0/856849693 shutdown_connections 2024-04-25T00:02:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:20.407+0000 7f5fe654c640 1 -- 172.21.15.110:0/856849693 wait complete. 2024-04-25T00:02:21.665 INFO:teuthology.orchestra.run.smithi110.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-04-25T00:02:21.668 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-04-25T00:02:21.934 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/: 2024-04-25T00:02:21.934 INFO:teuthology.orchestra.run.smithi110.stdout:total 2 2024-04-25T00:02:21.934 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 . 2024-04-25T00:02:21.934 INFO:teuthology.orchestra.run.smithi110.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 data 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data: 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:total 45 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 . 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 25 00:02 base 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 Apr 25 00:02 global 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 log 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_commit_ts 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_dynshmem 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Apr 25 00:02 pg_hba.conf 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Apr 25 00:02 pg_ident.conf 2024-04-25T00:02:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:02 pg_logical 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 pg_multixact 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_notify 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_replslot 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_serial 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_snapshots 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_stat 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_stat_tmp 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 pg_subtrans 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_tblspc 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_twophase 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 25 00:02 PG_VERSION 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:02 pg_wal 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 pg_xact 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Apr 25 00:02 postgresql.auto.conf 2024-04-25T00:02:21.936 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Apr 25 00:02 postgresql.conf 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base: 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:total 3 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 25 00:02 . 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 1 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 13493 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 13494 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base/1: 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:total 7778 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 . 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 25 00:02 .. 2024-04-25T00:02:21.937 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 112 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 113 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 1247 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1247_fsm 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1247_vm 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 25 00:02 1249 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1249_fsm 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1249_vm 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 25 00:02 1255 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1255_fsm 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1255_vm 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 25 00:02 1259 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1259_fsm 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1259_vm 2024-04-25T00:02:21.938 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 25 00:02 13344 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13344_fsm 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13344_vm 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13346 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13348 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13349_fsm 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349_vm 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13351 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13353 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13354_fsm 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354_vm 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13356 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13358 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359 2024-04-25T00:02:21.939 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13359_fsm 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359_vm 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13361 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13363 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1417 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1418 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 174 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 175 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2187 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2224 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2328 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2336 2024-04-25T00:02:21.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2337 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2579 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2600 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2600_fsm 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2600_vm 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2601_fsm 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601_vm 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2602 2024-04-25T00:02:21.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2602_fsm 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2602_vm 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2603 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2603_fsm 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2603_vm 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2604 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2605 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2605_fsm 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2605_vm 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2606_fsm 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606_vm 2024-04-25T00:02:21.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2607 2024-04-25T00:02:21.943 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2607_fsm 2024-04-25T00:02:21.943 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2607_vm 2024-04-25T00:02:21.943 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 25 00:02 2608 2024-04-25T00:02:21.943 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2608_fsm 2024-04-25T00:02:21.943 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2608_vm 2024-04-25T00:02:21.943 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 25 00:02 2609 2024-04-25T00:02:21.943 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2609_fsm 2024-04-25T00:02:21.943 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2609_vm 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2610 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2610_fsm 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2610_vm 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2611 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2612_fsm 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612_vm 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2613 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2615_fsm 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615_vm 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2616 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2616_fsm 2024-04-25T00:02:21.944 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2616_vm 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 25 00:02 2617 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2617_fsm 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2617_vm 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 25 00:02 2618 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2618_fsm 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2618_vm 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 25 00:02 2619 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2619_fsm 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2619_vm 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2620 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2650 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2651 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2652 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2653 2024-04-25T00:02:21.945 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2654 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2655 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2656 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2657 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 25 00:02 2658 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 2659 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2660 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2661 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2662 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2663 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2664 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2665 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2666 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2667 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2668 2024-04-25T00:02:21.946 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2669 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2670 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 25 00:02 2673 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 25 00:02 2674 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 25 00:02 2675 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2678 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2679 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2680 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2681 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2682 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2683 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2684 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2685 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2686 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2687 2024-04-25T00:02:21.947 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2688 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2689 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 25 00:02 2690 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 25 00:02 2691 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2692 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2693 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2696 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2699 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2701 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2702 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2703 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2704 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2753 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2753_fsm 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2753_vm 2024-04-25T00:02:21.948 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2754 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2755 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2756 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2757 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2830 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2831 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2832 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2833 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2834 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2835 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2836 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2837 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 25 00:02 2838 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2838_fsm 2024-04-25T00:02:21.949 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2838_vm 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2839 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840_fsm 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2840_vm 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2841 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2995 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2996 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3079_fsm 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079_vm 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3080 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3081 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3085 2024-04-25T00:02:21.950 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3118 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3119 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 25 00:02 3164 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3256 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3257 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3258 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3350 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3351 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3379 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3380 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3381 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3394 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3394_fsm 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3394_vm 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3395 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3429 2024-04-25T00:02:21.951 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3430 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3431 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3433 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3439 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3440 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3455 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 25 00:02 3456 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3456_fsm 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3456_vm 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3466 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3467 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3468 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3501 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3502 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3503 2024-04-25T00:02:21.952 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3534 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3541_fsm 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541_vm 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3542 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3574 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3575 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3576 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3596 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3597 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3598 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3599 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3600_fsm 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600_vm 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601 2024-04-25T00:02:21.953 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3601_fsm 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601_vm 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3602_fsm 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602_vm 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603_fsm 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3603_vm 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3604 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3605 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3606 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3607 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3608 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 3609 2024-04-25T00:02:21.954 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3712 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3764_fsm 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764_vm 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3766 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3767 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3997 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4143 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4144 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4145 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4146 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4147 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4148 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4149 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4150 2024-04-25T00:02:21.955 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4151 2024-04-25T00:02:21.956 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4152 2024-04-25T00:02:21.956 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4153 2024-04-25T00:02:21.956 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4154 2024-04-25T00:02:21.956 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4155 2024-04-25T00:02:21.956 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4156 2024-04-25T00:02:21.956 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4157 2024-04-25T00:02:21.956 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4158 2024-04-25T00:02:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:21 smithi188 ceph-mon[33285]: pgmap v360: 97 pgs: 97 active+clean; 40 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.3 MiB/s wr, 99 op/s 2024-04-25T00:02:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:21 smithi110 ceph-mon[29187]: pgmap v360: 97 pgs: 97 active+clean; 40 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.3 MiB/s wr, 99 op/s 2024-04-25T00:02:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:21 smithi173 ceph-mon[32402]: pgmap v360: 97 pgs: 97 active+clean; 40 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.3 MiB/s wr, 99 op/s 2024-04-25T00:02:22.149 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4159 2024-04-25T00:02:22.149 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4160 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4161 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4162 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4163 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4164 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4165 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4166 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4167 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4168 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4169 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4170 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4171 2024-04-25T00:02:22.150 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4172 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4173 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4174 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 5002 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 548 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 549 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6102 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6104 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6106 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6110 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6111 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6112 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6113 2024-04-25T00:02:22.151 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6117 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 826 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 827 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 828 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 25 00:02 pg_filenode.map 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 25 00:02 PG_VERSION 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:total 7778 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 . 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 25 00:02 .. 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 112 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 113 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 1247 2024-04-25T00:02:22.152 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1247_fsm 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1247_vm 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 25 00:02 1249 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1249_fsm 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1249_vm 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 25 00:02 1255 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1255_fsm 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1255_vm 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 25 00:02 1259 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1259_fsm 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1259_vm 2024-04-25T00:02:22.153 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 25 00:02 13344 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13344_fsm 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13344_vm 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13346 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13348 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13349_fsm 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349_vm 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13351 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13353 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13354_fsm 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354_vm 2024-04-25T00:02:22.154 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13356 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13358 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13359_fsm 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359_vm 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13361 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13363 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1417 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1418 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 174 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 175 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2187 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2224 2024-04-25T00:02:22.155 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2328 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2336 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2337 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2579 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2600 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2600_fsm 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2600_vm 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2601_fsm 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601_vm 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2602 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2602_fsm 2024-04-25T00:02:22.156 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2602_vm 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2603 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2603_fsm 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2603_vm 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2604 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2605 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2605_fsm 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2605_vm 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2606_fsm 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606_vm 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2607 2024-04-25T00:02:22.157 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2607_fsm 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2607_vm 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 25 00:02 2608 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2608_fsm 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2608_vm 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 25 00:02 2609 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2609_fsm 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2609_vm 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2610 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2610_fsm 2024-04-25T00:02:22.158 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2610_vm 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2611 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2612_fsm 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612_vm 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2613 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2615_fsm 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615_vm 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2616 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2616_fsm 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2616_vm 2024-04-25T00:02:22.159 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 25 00:02 2617 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2617_fsm 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2617_vm 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 25 00:02 2618 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2618_fsm 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2618_vm 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 25 00:02 2619 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2619_fsm 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2619_vm 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2620 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2650 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2651 2024-04-25T00:02:22.160 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2652 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2653 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2654 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2655 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2656 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2657 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 25 00:02 2658 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 2659 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2660 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2661 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2662 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2663 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2664 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2665 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2666 2024-04-25T00:02:22.161 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2667 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2668 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2669 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2670 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 25 00:02 2673 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 25 00:02 2674 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 25 00:02 2675 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2678 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2679 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2680 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2681 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2682 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2683 2024-04-25T00:02:22.162 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2684 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2685 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2686 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2687 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2688 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2689 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 25 00:02 2690 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 25 00:02 2691 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2692 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2693 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2696 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2699 2024-04-25T00:02:22.163 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2701 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2702 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2703 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2704 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2753 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2753_fsm 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2753_vm 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2754 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2755 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2756 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2757 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2830 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2831 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2832 2024-04-25T00:02:22.164 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2833 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2834 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2835 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2836 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2837 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 25 00:02 2838 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2838_fsm 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2838_vm 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2839 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840_fsm 2024-04-25T00:02:22.165 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2840_vm 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2841 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2995 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2996 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3079_fsm 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079_vm 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3080 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3081 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3085 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3118 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3119 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 25 00:02 3164 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3256 2024-04-25T00:02:22.166 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3257 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3258 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3350 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3351 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3379 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3380 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3381 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3394 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3394_fsm 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3394_vm 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3395 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3429 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3430 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3431 2024-04-25T00:02:22.167 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3433 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3439 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3440 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3455 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 25 00:02 3456 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3456_fsm 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3456_vm 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3466 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3467 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3468 2024-04-25T00:02:22.168 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3501 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3502 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3503 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3534 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3541_fsm 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541_vm 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3542 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3574 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3575 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3576 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3596 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3597 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3598 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3599 2024-04-25T00:02:22.169 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3600_fsm 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600_vm 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3601_fsm 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601_vm 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3602_fsm 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602_vm 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603 2024-04-25T00:02:22.170 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603_fsm 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3603_vm 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3604 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3605 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3606 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3607 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3608 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 3609 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3712 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3764_fsm 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764_vm 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3766 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3767 2024-04-25T00:02:22.171 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3997 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4143 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4144 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4145 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4146 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4147 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4148 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4149 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4150 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4151 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4152 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4153 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4154 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4155 2024-04-25T00:02:22.172 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4156 2024-04-25T00:02:22.173 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4157 2024-04-25T00:02:22.173 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4158 2024-04-25T00:02:22.173 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4159 2024-04-25T00:02:22.173 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4160 2024-04-25T00:02:22.173 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4161 2024-04-25T00:02:22.173 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4162 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4163 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4164 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4165 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4166 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4167 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4168 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4169 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4170 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4171 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4172 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4173 2024-04-25T00:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4174 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 5002 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 548 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 549 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6102 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6104 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6106 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6110 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6111 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6112 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6113 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6117 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 826 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 827 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 828 2024-04-25T00:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 25 00:02 pg_filenode.map 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 25 00:02 PG_VERSION 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:total 7778 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 . 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 25 00:02 .. 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 112 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 113 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 1247 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1247_fsm 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1247_vm 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 25 00:02 1249 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1249_fsm 2024-04-25T00:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1249_vm 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 25 00:02 1255 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1255_fsm 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1255_vm 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 25 00:02 1259 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1259_fsm 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1259_vm 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 25 00:02 13344 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13344_fsm 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13344_vm 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13346 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13348 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13349_fsm 2024-04-25T00:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349_vm 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13351 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13353 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13354_fsm 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354_vm 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13356 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13358 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13359_fsm 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359_vm 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13361 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13363 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1417 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1418 2024-04-25T00:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 174 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 175 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2187 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2224 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2328 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2336 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2337 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2579 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2600 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2600_fsm 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2600_vm 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2601_fsm 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601_vm 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2602 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2602_fsm 2024-04-25T00:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2602_vm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2603 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2603_fsm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2603_vm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2604 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2605 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2605_fsm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2605_vm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2606_fsm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606_vm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2607 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2607_fsm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2607_vm 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 25 00:02 2608 2024-04-25T00:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2608_fsm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2608_vm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 25 00:02 2609 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2609_fsm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2609_vm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2610 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2610_fsm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2610_vm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2611 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2612_fsm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612_vm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2613 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2615_fsm 2024-04-25T00:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615_vm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2616 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2616_fsm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2616_vm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 25 00:02 2617 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2617_fsm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2617_vm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 25 00:02 2618 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2618_fsm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2618_vm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 25 00:02 2619 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2619_fsm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2619_vm 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2620 2024-04-25T00:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2650 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2651 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2652 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2653 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2654 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2655 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2656 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2657 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 25 00:02 2658 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 2659 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2660 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2661 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2662 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2663 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2664 2024-04-25T00:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2665 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2666 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2667 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2668 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2669 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2670 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 25 00:02 2673 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 25 00:02 2674 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 25 00:02 2675 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2678 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2679 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2680 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2681 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2682 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2683 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2684 2024-04-25T00:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2685 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2686 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2687 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2688 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2689 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 25 00:02 2690 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 25 00:02 2691 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2692 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2693 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2696 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2699 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2701 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2702 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2703 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2704 2024-04-25T00:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2753 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2753_fsm 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2753_vm 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2754 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2755 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2756 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2757 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2830 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2831 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2832 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2833 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2834 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2835 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2836 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2837 2024-04-25T00:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 25 00:02 2838 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2838_fsm 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2838_vm 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2839 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840_fsm 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2840_vm 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2841 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2995 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2996 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3079_fsm 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079_vm 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3080 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3081 2024-04-25T00:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3085 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3118 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3119 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 25 00:02 3164 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3256 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3257 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3258 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3350 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3351 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3379 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3380 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3381 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3394 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3394_fsm 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3394_vm 2024-04-25T00:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3395 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3429 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3430 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3431 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3433 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3439 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3440 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3455 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 25 00:02 3456 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3456_fsm 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3456_vm 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3466 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3467 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3468 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3501 2024-04-25T00:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3502 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3503 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3534 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3541_fsm 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541_vm 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3542 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3574 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3575 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3576 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3596 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3597 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3598 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3599 2024-04-25T00:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3600_fsm 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600_vm 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3601_fsm 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601_vm 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3602_fsm 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602_vm 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603_fsm 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3603_vm 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3604 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3605 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3606 2024-04-25T00:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3607 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3608 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 3609 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3712 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3764_fsm 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764_vm 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3766 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3767 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3997 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4143 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4144 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4145 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4146 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4147 2024-04-25T00:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4148 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4149 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4150 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4151 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4152 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4153 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4154 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4155 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4156 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4157 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4158 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4159 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4160 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4161 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4162 2024-04-25T00:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4163 2024-04-25T00:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4164 2024-04-25T00:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4165 2024-04-25T00:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4166 2024-04-25T00:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4167 2024-04-25T00:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4168 2024-04-25T00:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4169 2024-04-25T00:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4170 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4171 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4172 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4173 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4174 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 5002 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 548 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 549 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6102 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6104 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6106 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6110 2024-04-25T00:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6111 2024-04-25T00:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6112 2024-04-25T00:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6113 2024-04-25T00:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6117 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 826 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 827 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 828 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 25 00:02 pg_filenode.map 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 25 00:02 PG_VERSION 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/global: 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:total 538 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 Apr 25 00:02 . 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1213 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1213_fsm 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1213_vm 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1214 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1214_fsm 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1214_vm 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 1232 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 1233 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1260 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1260_fsm 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1260_vm 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1261 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1261_fsm 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1261_vm 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1262 2024-04-25T00:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1262_fsm 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1262_vm 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2396 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2396_fsm 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2396_vm 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2397 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2671 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2672 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2676 2024-04-25T00:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2677 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2694 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2695 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2697 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2698 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2846 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2847 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2964 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2965 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2966 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2967 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3592 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3593 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4060 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4061 2024-04-25T00:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4175 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4176 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4177 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4178 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4181 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4182 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4183 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4184 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4185 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4186 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6000 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6001 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6002 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6100 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6114 2024-04-25T00:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6115 2024-04-25T00:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 pg_control 2024-04-25T00:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 25 00:02 pg_filenode.map 2024-04-25T00:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/log: 2024-04-25T00:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:total 3 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:02 . 2024-04-25T00:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 mappings 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:02 replorigin_checkpoint 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 snapshots 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:02 .. 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:02 .. 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:total 2 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 . 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 members 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 offsets 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:total 9 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 . 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 .. 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 0000 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-04-25T00:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout:total 9 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 . 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 .. 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 0000 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-04-25T00:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-04-25T00:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-04-25T00:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout:total 9 2024-04-25T00:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 . 2024-04-25T00:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 0000 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:total 16386 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:02 . 2024-04-25T00:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:02 000000010000000000000001 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 archive_status 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:02 .. 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:total 9 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 . 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 25 00:02 .. 2024-04-25T00:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 0000 2024-04-25T00:02:22.471 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-04-25T00:02:22.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-04-25T00:02:22.710 INFO:teuthology.orchestra.run.smithi110.stderr:dropping old tables... 2024-04-25T00:02:22.716 INFO:teuthology.orchestra.run.smithi110.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-04-25T00:02:22.717 INFO:teuthology.orchestra.run.smithi110.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-04-25T00:02:22.717 INFO:teuthology.orchestra.run.smithi110.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-04-25T00:02:22.717 INFO:teuthology.orchestra.run.smithi110.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-04-25T00:02:22.717 INFO:teuthology.orchestra.run.smithi110.stderr:creating tables... 2024-04-25T00:02:22.762 INFO:teuthology.orchestra.run.smithi110.stderr:generating data (client-side)... 2024-04-25T00:02:23.237 INFO:teuthology.orchestra.run.smithi110.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.45 s, remaining 14.04 s) 2024-04-25T00:02:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:23 smithi188 ceph-mon[33285]: pgmap v361: 97 pgs: 97 active+clean; 45 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 4.9 MiB/s wr, 226 op/s 2024-04-25T00:02:24.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:23 smithi110 ceph-mon[29187]: pgmap v361: 97 pgs: 97 active+clean; 45 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 4.9 MiB/s wr, 226 op/s 2024-04-25T00:02:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:23 smithi173 ceph-mon[32402]: pgmap v361: 97 pgs: 97 active+clean; 45 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 4.9 MiB/s wr, 226 op/s 2024-04-25T00:02:24.294 INFO:teuthology.orchestra.run.smithi110.stderr:200000 of 3200000 tuples (6%) done (elapsed 1.51 s, remaining 22.65 s) 2024-04-25T00:02:25.471 INFO:teuthology.orchestra.run.smithi110.stderr:300000 of 3200000 tuples (9%) done (elapsed 2.69 s, remaining 25.98 s) 2024-04-25T00:02:25.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:25 smithi188 ceph-mon[33285]: pgmap v362: 97 pgs: 97 active+clean; 45 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 5.4 MiB/s wr, 321 op/s 2024-04-25T00:02:25.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:25 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:02:25.962 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:25 smithi173 ceph-mon[32402]: pgmap v362: 97 pgs: 97 active+clean; 45 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 5.4 MiB/s wr, 321 op/s 2024-04-25T00:02:25.962 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:25 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:02:25.964 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:25 smithi110 ceph-mon[29187]: pgmap v362: 97 pgs: 97 active+clean; 45 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 5.4 MiB/s wr, 321 op/s 2024-04-25T00:02:25.964 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:25 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:02:26.188 INFO:teuthology.orchestra.run.smithi110.stderr:400000 of 3200000 tuples (12%) done (elapsed 3.40 s, remaining 23.83 s) 2024-04-25T00:02:27.284 INFO:teuthology.orchestra.run.smithi110.stderr:500000 of 3200000 tuples (15%) done (elapsed 4.50 s, remaining 24.30 s) 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: pgmap v363: 97 pgs: 97 active+clean; 53 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 4.6 MiB/s wr, 314 op/s 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:02:27.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: pgmap v363: 97 pgs: 97 active+clean; 53 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 4.6 MiB/s wr, 314 op/s 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:02:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: pgmap v363: 97 pgs: 97 active+clean; 53 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 4.6 MiB/s wr, 314 op/s 2024-04-25T00:02:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:02:28.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:02:28.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:02:28.073 INFO:teuthology.orchestra.run.smithi110.stderr:600000 of 3200000 tuples (18%) done (elapsed 5.29 s, remaining 22.92 s) 2024-04-25T00:02:29.152 INFO:teuthology.orchestra.run.smithi110.stderr:700000 of 3200000 tuples (21%) done (elapsed 6.37 s, remaining 22.75 s) 2024-04-25T00:02:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:29 smithi173 ceph-mon[32402]: pgmap v364: 97 pgs: 97 active+clean; 118 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 15 MiB/s wr, 304 op/s 2024-04-25T00:02:29.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:29 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:29 smithi188 ceph-mon[33285]: pgmap v364: 97 pgs: 97 active+clean; 118 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 15 MiB/s wr, 304 op/s 2024-04-25T00:02:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:29 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:29 smithi110 ceph-mon[29187]: pgmap v364: 97 pgs: 97 active+clean; 118 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 15 MiB/s wr, 304 op/s 2024-04-25T00:02:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:29 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:30.218 INFO:teuthology.orchestra.run.smithi110.stderr:800000 of 3200000 tuples (25%) done (elapsed 7.43 s, remaining 22.30 s) 2024-04-25T00:02:31.011 INFO:teuthology.orchestra.run.smithi110.stderr:900000 of 3200000 tuples (28%) done (elapsed 8.23 s, remaining 21.02 s) 2024-04-25T00:02:31.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:31 smithi173 ceph-mon[32402]: pgmap v365: 97 pgs: 97 active+clean; 175 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 22 MiB/s wr, 278 op/s 2024-04-25T00:02:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:31 smithi188 ceph-mon[33285]: pgmap v365: 97 pgs: 97 active+clean; 175 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 22 MiB/s wr, 278 op/s 2024-04-25T00:02:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:31 smithi110 ceph-mon[29187]: pgmap v365: 97 pgs: 97 active+clean; 175 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 22 MiB/s wr, 278 op/s 2024-04-25T00:02:32.049 INFO:teuthology.orchestra.run.smithi110.stderr:1000000 of 3200000 tuples (31%) done (elapsed 9.27 s, remaining 20.38 s) 2024-04-25T00:02:33.126 INFO:teuthology.orchestra.run.smithi110.stderr:1100000 of 3200000 tuples (34%) done (elapsed 10.34 s, remaining 19.74 s) 2024-04-25T00:02:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 -- 172.21.15.110:0/3469642717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 msgr2=0x7fcce4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/3469642717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 0x7fcce4108ae0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009ca0 tx=0x7fccdc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 -- 172.21.15.110:0/3469642717 shutdown_connections 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/3469642717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 0x7fcce4108ae0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/3469642717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcce4101890 0x7fcce4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/3469642717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcce41006a0 0x7fcce4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 -- 172.21.15.110:0/3469642717 >> 172.21.15.110:0/3469642717 conn(0x7fcce40fbc10 msgr2=0x7fcce40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 -- 172.21.15.110:0/3469642717 shutdown_connections 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.501+0000 7fcceb8fc640 1 -- 172.21.15.110:0/3469642717 wait complete. 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcceb8fc640 1 Processor -- start 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcceb8fc640 1 -- start start 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcceb8fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcce41006a0 0x7fcce4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcceb8fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcce4101890 0x7fcce4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcceb8fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 0x7fcce419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcceb8fc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcce410c1b0 con 0x7fcce4102250 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcceb8fc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcce410c030 con 0x7fcce4101890 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcceb8fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcce410c330 con 0x7fcce41006a0 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcce9e99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 0x7fcce419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcce9e99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 0x7fcce419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59740/0 (socket says 172.21.15.110:59740) 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.502+0000 7fcce9e99640 1 -- 172.21.15.110:0/1747234624 learned_addr learned my addr 172.21.15.110:0/1747234624 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcce9698640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcce41006a0 0x7fcce4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcce8e97640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcce4101890 0x7fcce4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcce9e99640 1 -- 172.21.15.110:0/1747234624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcce41006a0 msgr2=0x7fcce4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcce41006a0 0x7fcce4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcce9e99640 1 -- 172.21.15.110:0/1747234624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcce4101890 msgr2=0x7fcce4197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcce4101890 0x7fcce4197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcce9e99640 1 -- 172.21.15.110:0/1747234624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce419f490 con 0x7fcce4102250 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 0x7fcce419ee20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009dc0 tx=0x7fccdc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fccda7fc640 1 -- 172.21.15.110:0/1747234624 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc032ec0 con 0x7fcce4102250 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fccda7fc640 1 -- 172.21.15.110:0/1747234624 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccdc0044f0 con 0x7fcce4102250 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcceb8fc640 1 -- 172.21.15.110:0/1747234624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce4102ab0 con 0x7fcce4102250 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fcceb8fc640 1 -- 172.21.15.110:0/1747234624 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce4103050 con 0x7fcce4102250 2024-04-25T00:02:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.503+0000 7fccda7fc640 1 -- 172.21.15.110:0/1747234624 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc030d40 con 0x7fcce4102250 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.506+0000 7fccda7fc640 1 -- 172.21.15.110:0/1747234624 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fccdc0046a0 con 0x7fcce4102250 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.506+0000 7fccee60a740 1 Processor -- start 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccee60a740 1 -- start start 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccda7fc640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccd00a2e50 0x7fccd00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccee60a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373afe50 0x5651373b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccee60a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b0840 0x5651373b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccee60a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b59b0 0x5651373b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccee60a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x565137313cb0 con 0x5651373b0840 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccee60a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5651372bc5b0 con 0x5651373b59b0 2024-04-25T00:02:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccee60a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56513730fc70 con 0x5651373afe50 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce8e97640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccd00a2e50 0x7fccd00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fccda7fc640 1 -- 172.21.15.110:0/1747234624 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fccdc0f0540 con 0x7fcce4102250 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce8e97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b0840 0x5651373b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce8e97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b0840 0x5651373b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59744/0 (socket says 172.21.15.110:59744) 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce9e99640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b59b0 0x5651373b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce8e97640 1 -- 172.21.15.110:0/417416452 learned_addr learned my addr 172.21.15.110:0/417416452 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce9698640 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373afe50 0x5651373b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce9e99640 1 -- 172.21.15.110:0/417416452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373afe50 msgr2=0x5651373b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce9e99640 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373afe50 0x5651373b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce9e99640 1 -- 172.21.15.110:0/417416452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b0840 msgr2=0x5651373b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce9e99640 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b0840 0x5651373b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce9e99640 1 -- 172.21.15.110:0/417416452 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565137028210 con 0x5651373b59b0 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.507+0000 7fcce8e97640 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b0840 0x5651373b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.508+0000 7fcce8e97640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccd00a2e50 0x7fccd00a5310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fccd4013990 tx=0x7fccd400bb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.508+0000 7fcce9e99640 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b59b0 0x5651373b7e40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fccdc031970 tx=0x7fccdc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de72e4e9185d1caa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.508+0000 7fcccf7fe640 1 -- 172.21.15.110:0/417416452 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc039c40 con 0x5651373b59b0 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.508+0000 7fcccf7fe640 1 -- 172.21.15.110:0/417416452 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccdc036040 con 0x5651373b59b0 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.508+0000 7fcccf7fe640 1 -- 172.21.15.110:0/417416452 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc004b40 con 0x5651373b59b0 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- 172.21.15.110:0/417416452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b59b0 msgr2=0x5651373b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b59b0 0x5651373b7e40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fccdc031970 tx=0x7fccdc0095c0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- 172.21.15.110:0/417416452 shutdown_connections 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b59b0 0x5651373b7e40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b0840 0x5651373b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 --2- 172.21.15.110:0/417416452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373afe50 0x5651373b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- 172.21.15.110:0/417416452 >> 172.21.15.110:0/417416452 conn(0x5651373ab4c0 msgr2=0x5651373ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- 172.21.15.110:0/417416452 shutdown_connections 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- 172.21.15.110:0/417416452 wait complete. 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- start start 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 Processor -- start 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5651372bc5b0 con 0x5651373b59b0 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5651372cfb50 con 0x5651373afe50 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5651373b8720 con 0x5651373b0840 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fccee60a740 10 client.0 _ll_register_callbacks cb 0x56513730aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.509+0000 7fcce9e99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9e99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59754/0 (socket says 172.21.15.110:59754) 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9e99640 1 -- 172.21.15.110:0/1986295702 learned_addr learned my addr 172.21.15.110:0/1986295702 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9698640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce8e97640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9e99640 1 -- 172.21.15.110:0/1986295702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 msgr2=0x5651373ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9e99640 1 -- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 msgr2=0x5651373b9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373b9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9e99640 1 -- 172.21.15.110:0/1986295702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651370f4620 con 0x5651373b59b0 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce8e97640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9698640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373c1400 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0ee3e0 tx=0x7fccdc0bcdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6743ddab94fc82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fccccff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fccccff9640 1 -- 172.21.15.110:0/1986295702 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0d9070 con 0x5651373b59b0 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fccccff9640 1 -- 172.21.15.110:0/1986295702 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fccdc0305c0 con 0x5651373b59b0 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fccee60a740 1 -- 172.21.15.110:0/1986295702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccdc036040 con 0x5651373b59b0 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fccccff9640 1 -- 172.21.15.110:0/1986295702 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fccdc03b410 con 0x5651373b59b0 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fccee60a740 10 client.15309 fetch_fsmap learned FSMap version 23 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.510+0000 7fccee60a740 1 -- 172.21.15.110:0/1986295702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5651373c4770 con 0x5651373b59b0 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccccff9640 1 -- 172.21.15.110:0/1986295702 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccdc036040 con 0x5651373b59b0 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccccff9640 1 -- 172.21.15.110:0/1986295702 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 23) ==== 8342+0+0 (secure 0 0 0) 0x7fccdc0d2030 con 0x5651373b59b0 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccee60a740 10 client.15309 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccee60a740 10 client.15309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccee60a740 20 client.15309 populate_metadata read hostname 'smithi110' 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccee60a740 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651373c5b80 0x5651373c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccee60a740 4 client.15309 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccee60a740 1 -- 172.21.15.110:0/1986295702 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651373211f0 con 0x5651373c5b80 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fcce9698640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651373c5b80 0x5651373c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fccccff9640 10 client.15309 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.511+0000 7fcce9698640 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651373c5b80 0x5651373c7fc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9483b4ce53d4caf server_cookie=aa6850ef23946e12 in_seq=0 out_seq=0 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccccff9640 1 -- 172.21.15.110:0/1986295702 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651373211f0 con 0x5651373c5b80 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccccff9640 10 client.15309 handle_command_reply: tid=0 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 client.15309 shutdown 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 -- 172.21.15.110:0/1986295702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 msgr2=0x5651373c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373c1400 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0ee3e0 tx=0x7fccdc0bcdd0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 -- 172.21.15.110:0/1986295702 shutdown_connections 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651373c5b80 0x5651373c7fc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccccff9640 0 client.15309 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373c1400 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 --2- 172.21.15.110:0/1986295702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.512+0000 7fccee60a740 1 -- 172.21.15.110:0/1986295702 >> 172.21.15.110:0/1986295702 conn(0x5651373ab4c0 msgr2=0x5651373aa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.513+0000 7fccee60a740 1 -- 172.21.15.110:0/1986295702 shutdown_connections 2024-04-25T00:02:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.513+0000 7fccee60a740 1 -- 172.21.15.110:0/1986295702 wait complete. 2024-04-25T00:02:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.513+0000 7fccee60a740 20 client.15309 trim_cache size 0 max 16384 2024-04-25T00:02:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fccee60a740 1 Processor -- start 2024-04-25T00:02:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fccee60a740 1 -- start start 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fccee60a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373afe50 0x5651373b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fccee60a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b0840 0x5651373c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fccee60a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b59b0 0x5651373ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fccee60a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8010c40 con 0x5651373afe50 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fccee60a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fccc800e510 con 0x5651373b0840 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fccee60a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fccc800c360 con 0x5651373b59b0 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.529+0000 7fcce9698640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373afe50 0x5651373b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9698640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373afe50 0x5651373b9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59762/0 (socket says 172.21.15.110:59762) 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9698640 1 -- 172.21.15.110:0/2381204321 learned_addr learned my addr 172.21.15.110:0/2381204321 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce8e97640 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b0840 0x5651373c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9e99640 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b59b0 0x5651373ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9698640 1 -- 172.21.15.110:0/2381204321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b59b0 msgr2=0x5651373ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9698640 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b59b0 0x5651373ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9698640 1 -- 172.21.15.110:0/2381204321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b0840 msgr2=0x5651373c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9698640 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b0840 0x5651373c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9698640 1 -- 172.21.15.110:0/2381204321 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651373b8810 con 0x5651373afe50 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9e99640 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b59b0 0x5651373ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce8e97640 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b0840 0x5651373c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fcce9698640 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373afe50 0x5651373b9880 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcce41016f0 tx=0x7fcce000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff5e487910816f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fccceffd640 1 -- 172.21.15.110:0/2381204321 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0003e00 con 0x5651373afe50 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fccceffd640 1 -- 172.21.15.110:0/2381204321 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce00054d0 con 0x5651373afe50 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.530+0000 7fccceffd640 1 -- 172.21.15.110:0/2381204321 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce00201b0 con 0x5651373afe50 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 -- 172.21.15.110:0/2381204321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373afe50 msgr2=0x5651373b9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373afe50 0x5651373b9880 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcce41016f0 tx=0x7fcce000bed0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 -- 172.21.15.110:0/2381204321 shutdown_connections 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b59b0 0x5651373ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373b0840 0x5651373c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 --2- 172.21.15.110:0/2381204321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373afe50 0x5651373b9880 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 -- 172.21.15.110:0/2381204321 >> 172.21.15.110:0/2381204321 conn(0x5651373ab4c0 msgr2=0x5651373b9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 -- 172.21.15.110:0/2381204321 shutdown_connections 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 -- 172.21.15.110:0/2381204321 wait complete. 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 -- start start 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.531+0000 7fccee60a740 1 Processor -- start 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fccee60a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373be6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fccee60a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fccee60a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fccee60a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fccc800c9d0 con 0x5651373b59b0 2024-04-25T00:02:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fccee60a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fccc800de40 con 0x5651373afe50 2024-04-25T00:02:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fccee60a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fccc800d0a0 con 0x5651373b0840 2024-04-25T00:02:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fccee60a740 10 client.0 _ll_register_callbacks cb 0x7fccd40134a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373bfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59770/0 (socket says 172.21.15.110:59770) 2024-04-25T00:02:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 -- 172.21.15.110:0/1294794813 learned_addr learned my addr 172.21.15.110:0/1294794813 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce8e97640 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9698640 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373be6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 -- 172.21.15.110:0/1294794813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 msgr2=0x5651373bebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373bebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 -- 172.21.15.110:0/1294794813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 msgr2=0x5651373be6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373be6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 -- 172.21.15.110:0/1294794813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651373c01e0 con 0x5651373b59b0 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce8e97640 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373bebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fcce9e99640 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373bfbf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fccdc02f3b0 tx=0x7fccdc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8194e842a7e77b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.532+0000 7fccccff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccccff9640 1 -- 172.21.15.110:0/1294794813 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0d9030 con 0x5651373b59b0 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccccff9640 1 -- 172.21.15.110:0/1294794813 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fccdc0f00c0 con 0x5651373b59b0 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 1 -- 172.21.15.110:0/1294794813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce00054d0 con 0x5651373b59b0 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccccff9640 1 -- 172.21.15.110:0/1294794813 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fccdc0bd4d0 con 0x5651373b59b0 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 10 client.15315 fetch_fsmap learned FSMap version 23 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 1 -- 172.21.15.110:0/1294794813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5651373c01e0 con 0x5651373b59b0 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccccff9640 1 -- 172.21.15.110:0/1294794813 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce00054d0 con 0x5651373b59b0 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccccff9640 1 -- 172.21.15.110:0/1294794813 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 23) ==== 8342+0+0 (secure 0 0 0) 0x7fccdc0d2070 con 0x5651373b59b0 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 10 client.15315 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 10 client.15315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 20 client.15315 populate_metadata read hostname 'smithi110' 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651373c5b80 0x56513742e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 4 client.15315 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:02:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.533+0000 7fccee60a740 1 -- 172.21.15.110:0/1294794813 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56513731b540 con 0x5651373c5b80 2024-04-25T00:02:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.534+0000 7fcce9698640 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651373c5b80 0x56513742e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.534+0000 7fccccff9640 10 client.15315 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:33.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.534+0000 7fcce9698640 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651373c5b80 0x56513742e2d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b592e931a9cec5e5 server_cookie=747a5f2c4a158b7 in_seq=0 out_seq=0 2024-04-25T00:02:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccccff9640 1 -- 172.21.15.110:0/1294794813 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56513731b540 con 0x5651373c5b80 2024-04-25T00:02:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccccff9640 10 client.15315 handle_command_reply: tid=0 2024-04-25T00:02:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 client.15315 shutdown 2024-04-25T00:02:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 -- 172.21.15.110:0/1294794813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 msgr2=0x5651373bfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373bfbf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fccdc02f3b0 tx=0x7fccdc0095c0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 -- 172.21.15.110:0/1294794813 shutdown_connections 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651373c5b80 0x56513742e2d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651373b59b0 0x5651373bfbf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651373b0840 0x5651373bebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccccff9640 0 client.15315 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 --2- 172.21.15.110:0/1294794813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651373afe50 0x5651373be6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.590+0000 7fccee60a740 1 -- 172.21.15.110:0/1294794813 >> 172.21.15.110:0/1294794813 conn(0x5651373ab4c0 msgr2=0x5651373a9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.591+0000 7fccee60a740 1 -- 172.21.15.110:0/1294794813 shutdown_connections 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.591+0000 7fccee60a740 1 -- 172.21.15.110:0/1294794813 wait complete. 2024-04-25T00:02:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.591+0000 7fccee60a740 20 client.15315 trim_cache size 0 max 16384 2024-04-25T00:02:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:02:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:02:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:02:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:02:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.591+0000 7fcceb8fc640 1 -- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccd00a2e50 msgr2=0x7fccd00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.591+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccd00a2e50 0x7fccd00a5310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fccd4013990 tx=0x7fccd400bb70 comp rx=0 tx=0).stop 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.591+0000 7fcceb8fc640 1 -- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 msgr2=0x7fcce419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.591+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 0x7fcce419ee20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009dc0 tx=0x7fccdc003ba0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.592+0000 7fcceb8fc640 1 -- 172.21.15.110:0/1747234624 shutdown_connections 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.592+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccd00a2e50 0x7fccd00a5310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.592+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcce4102250 0x7fcce419ee20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.592+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcce4101890 0x7fcce4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.592+0000 7fcceb8fc640 1 --2- 172.21.15.110:0/1747234624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcce41006a0 0x7fcce4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.592+0000 7fcceb8fc640 1 -- 172.21.15.110:0/1747234624 >> 172.21.15.110:0/1747234624 conn(0x7fcce40fbc10 msgr2=0x7fcce40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.592+0000 7fcceb8fc640 1 -- 172.21.15.110:0/1747234624 shutdown_connections 2024-04-25T00:02:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:33.592+0000 7fcceb8fc640 1 -- 172.21.15.110:0/1747234624 wait complete. 2024-04-25T00:02:33.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:33 smithi173 ceph-mon[32402]: pgmap v366: 97 pgs: 97 active+clean; 217 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 28 MiB/s wr, 278 op/s 2024-04-25T00:02:33.916 INFO:teuthology.orchestra.run.smithi110.stderr:1200000 of 3200000 tuples (37%) done (elapsed 11.13 s, remaining 18.55 s) 2024-04-25T00:02:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:33 smithi188 ceph-mon[33285]: pgmap v366: 97 pgs: 97 active+clean; 217 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 28 MiB/s wr, 278 op/s 2024-04-25T00:02:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:33 smithi110 ceph-mon[29187]: pgmap v366: 97 pgs: 97 active+clean; 217 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 28 MiB/s wr, 278 op/s 2024-04-25T00:02:34.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:34 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:34.938 INFO:teuthology.orchestra.run.smithi110.stderr:1300000 of 3200000 tuples (40%) done (elapsed 12.15 s, remaining 17.76 s) 2024-04-25T00:02:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:34 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:34 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:02:35.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:35 smithi173 ceph-mon[32402]: pgmap v367: 97 pgs: 97 active+clean; 277 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 37 MiB/s wr, 161 op/s 2024-04-25T00:02:36.016 INFO:teuthology.orchestra.run.smithi110.stderr:1400000 of 3200000 tuples (43%) done (elapsed 13.23 s, remaining 17.01 s) 2024-04-25T00:02:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:35 smithi188 ceph-mon[33285]: pgmap v367: 97 pgs: 97 active+clean; 277 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 37 MiB/s wr, 161 op/s 2024-04-25T00:02:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:35 smithi110 ceph-mon[29187]: pgmap v367: 97 pgs: 97 active+clean; 277 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 37 MiB/s wr, 161 op/s 2024-04-25T00:02:36.773 INFO:teuthology.orchestra.run.smithi110.stderr:1500000 of 3200000 tuples (46%) done (elapsed 13.99 s, remaining 15.85 s) 2024-04-25T00:02:37.837 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:37 smithi188 ceph-mon[33285]: pgmap v368: 97 pgs: 97 active+clean; 289 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 39 MiB/s wr, 58 op/s 2024-04-25T00:02:37.847 INFO:teuthology.orchestra.run.smithi110.stderr:1600000 of 3200000 tuples (50%) done (elapsed 15.06 s, remaining 15.06 s) 2024-04-25T00:02:37.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:37 smithi173 ceph-mon[32402]: pgmap v368: 97 pgs: 97 active+clean; 289 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 39 MiB/s wr, 58 op/s 2024-04-25T00:02:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:37 smithi110 ceph-mon[29187]: pgmap v368: 97 pgs: 97 active+clean; 289 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 39 MiB/s wr, 58 op/s 2024-04-25T00:02:38.527 INFO:teuthology.orchestra.run.smithi110.stderr:1700000 of 3200000 tuples (53%) done (elapsed 15.74 s, remaining 13.89 s) 2024-04-25T00:02:39.650 INFO:teuthology.orchestra.run.smithi110.stderr:1800000 of 3200000 tuples (56%) done (elapsed 16.87 s, remaining 13.12 s) 2024-04-25T00:02:39.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:39 smithi173 ceph-mon[32402]: pgmap v369: 97 pgs: 97 active+clean; 370 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 51 MiB/s wr, 51 op/s 2024-04-25T00:02:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:39 smithi188 ceph-mon[33285]: pgmap v369: 97 pgs: 97 active+clean; 370 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 51 MiB/s wr, 51 op/s 2024-04-25T00:02:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:39 smithi110 ceph-mon[29187]: pgmap v369: 97 pgs: 97 active+clean; 370 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 51 MiB/s wr, 51 op/s 2024-04-25T00:02:40.689 INFO:teuthology.orchestra.run.smithi110.stderr:1900000 of 3200000 tuples (59%) done (elapsed 17.90 s, remaining 12.25 s) 2024-04-25T00:02:41.475 INFO:teuthology.orchestra.run.smithi110.stderr:2000000 of 3200000 tuples (62%) done (elapsed 18.69 s, remaining 11.21 s) 2024-04-25T00:02:41.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:41 smithi173 ceph-mon[32402]: pgmap v370: 97 pgs: 97 active+clean; 398 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 46 MiB/s wr, 50 op/s 2024-04-25T00:02:42.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:41 smithi188 ceph-mon[33285]: pgmap v370: 97 pgs: 97 active+clean; 398 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 46 MiB/s wr, 50 op/s 2024-04-25T00:02:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:41 smithi110 ceph-mon[29187]: pgmap v370: 97 pgs: 97 active+clean; 398 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 46 MiB/s wr, 50 op/s 2024-04-25T00:02:42.539 INFO:teuthology.orchestra.run.smithi110.stderr:2100000 of 3200000 tuples (65%) done (elapsed 19.75 s, remaining 10.35 s) 2024-04-25T00:02:43.597 INFO:teuthology.orchestra.run.smithi110.stderr:2200000 of 3200000 tuples (68%) done (elapsed 20.81 s, remaining 9.46 s) 2024-04-25T00:02:43.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:43 smithi173 ceph-mon[32402]: pgmap v371: 97 pgs: 97 active+clean; 442 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 MiB/s wr, 50 op/s 2024-04-25T00:02:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:43 smithi188 ceph-mon[33285]: pgmap v371: 97 pgs: 97 active+clean; 442 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 MiB/s wr, 50 op/s 2024-04-25T00:02:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:43 smithi110 ceph-mon[29187]: pgmap v371: 97 pgs: 97 active+clean; 442 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 MiB/s wr, 50 op/s 2024-04-25T00:02:44.370 INFO:teuthology.orchestra.run.smithi110.stderr:2300000 of 3200000 tuples (71%) done (elapsed 21.59 s, remaining 8.45 s) 2024-04-25T00:02:45.450 INFO:teuthology.orchestra.run.smithi110.stderr:2400000 of 3200000 tuples (75%) done (elapsed 22.67 s, remaining 7.56 s) 2024-04-25T00:02:45.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:45 smithi173 ceph-mon[32402]: pgmap v372: 97 pgs: 97 active+clean; 500 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 MiB/s wr, 53 op/s 2024-04-25T00:02:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:45 smithi188 ceph-mon[33285]: pgmap v372: 97 pgs: 97 active+clean; 500 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 MiB/s wr, 53 op/s 2024-04-25T00:02:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:45 smithi110 ceph-mon[29187]: pgmap v372: 97 pgs: 97 active+clean; 500 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 MiB/s wr, 53 op/s 2024-04-25T00:02:46.535 INFO:teuthology.orchestra.run.smithi110.stderr:2500000 of 3200000 tuples (78%) done (elapsed 23.75 s, remaining 6.65 s) 2024-04-25T00:02:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 -- 172.21.15.110:0/3833650887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 msgr2=0x7f7974101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 --2- 172.21.15.110:0/3833650887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 0x7f7974101d10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7960009900 tx=0x7f796002a410 comp rx=0 tx=0).stop 2024-04-25T00:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 -- 172.21.15.110:0/3833650887 shutdown_connections 2024-04-25T00:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 --2- 172.21.15.110:0/3833650887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7974102250 0x7f7974108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 --2- 172.21.15.110:0/3833650887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 0x7f7974101d10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 --2- 172.21.15.110:0/3833650887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79741006a0 0x7f7974100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 -- 172.21.15.110:0/3833650887 >> 172.21.15.110:0/3833650887 conn(0x7f79740fbc10 msgr2=0x7f79740fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 -- 172.21.15.110:0/3833650887 shutdown_connections 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.694+0000 7f797aef7640 1 -- 172.21.15.110:0/3833650887 wait complete. 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f797aef7640 1 Processor -- start 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f797aef7640 1 -- start start 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f797aef7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79741006a0 0x7f7974069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f797aef7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 0x7f797419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f797aef7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7974102250 0x7f797419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f797aef7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f797410c1d0 con 0x7f7974101890 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f797aef7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f797410c050 con 0x7f79741006a0 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f797aef7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f797410c350 con 0x7f7974102250 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7973fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 0x7f797419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7973fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 0x7f797419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49884/0 (socket says 172.21.15.110:49884) 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7973fff640 1 -- 172.21.15.110:0/1213810950 learned_addr learned my addr 172.21.15.110:0/1213810950 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7978c93640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79741006a0 0x7f7974069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7979494640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7974102250 0x7f797419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7973fff640 1 -- 172.21.15.110:0/1213810950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7974102250 msgr2=0x7f797419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7973fff640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7974102250 0x7f797419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7973fff640 1 -- 172.21.15.110:0/1213810950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79741006a0 msgr2=0x7f7974069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7973fff640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79741006a0 0x7f7974069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.695+0000 7f7973fff640 1 -- 172.21.15.110:0/1213810950 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797419f5b0 con 0x7f7974101890 2024-04-25T00:02:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.696+0000 7f7973fff640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 0x7f797419c700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7960005940 tx=0x7f79600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.696+0000 7f7971ffb640 1 -- 172.21.15.110:0/1213810950 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796003f030 con 0x7f7974101890 2024-04-25T00:02:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.696+0000 7f797aef7640 1 -- 172.21.15.110:0/1213810950 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797419f840 con 0x7f7974101890 2024-04-25T00:02:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.696+0000 7f7971ffb640 1 -- 172.21.15.110:0/1213810950 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7960003cb0 con 0x7f7974101890 2024-04-25T00:02:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.696+0000 7f7971ffb640 1 -- 172.21.15.110:0/1213810950 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960035070 con 0x7f7974101890 2024-04-25T00:02:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.696+0000 7f797aef7640 1 -- 172.21.15.110:0/1213810950 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797419fd80 con 0x7f7974101890 2024-04-25T00:02:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f7971ffb640 1 -- 172.21.15.110:0/1213810950 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7960042030 con 0x7f7974101890 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f797dbfd740 1 Processor -- start 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f797dbfd740 1 -- start start 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a43500680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43505750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a43508250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f797dbfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557a4345fcb0 con 0x557a43500260 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f797dbfd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557a434085b0 con 0x557a43505dc0 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f797dbfd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557a4345bc70 con 0x557a43500c50 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f7979494640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a43500680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f7979494640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a43500680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49886/0 (socket says 172.21.15.110:49886) 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f7979494640 1 -- 172.21.15.110:0/169537739 learned_addr learned my addr 172.21.15.110:0/169537739 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f7973fff640 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a43508250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f7971ffb640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79640a2c30 0x7f79640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.699+0000 7f7978c93640 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43505750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7979494640 1 -- 172.21.15.110:0/169537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 msgr2=0x557a43505750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7979494640 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43505750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7979494640 1 -- 172.21.15.110:0/169537739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 msgr2=0x557a43508250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7979494640 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a43508250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7979494640 1 -- 172.21.15.110:0/169537739 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a43174210 con 0x557a43500260 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7973fff640 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a43508250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7978c93640 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43505750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7971ffb640 1 -- 172.21.15.110:0/1213810950 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f79600ef610 con 0x7f7974101890 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7979494640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79640a2c30 0x7f79640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7979494640 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a43500680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f796c0103f0 tx=0x7f796c0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765e6edc8a09264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f795effd640 1 -- 172.21.15.110:0/169537739 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c01c900 con 0x557a43500260 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f795effd640 1 -- 172.21.15.110:0/169537739 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f796c0113e0 con 0x557a43500260 2024-04-25T00:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f795effd640 1 -- 172.21.15.110:0/169537739 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c01b630 con 0x557a43500260 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.700+0000 7f7979494640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79640a2c30 0x7f79640a50f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f796c020000 tx=0x7f796c01d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 -- 172.21.15.110:0/169537739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 msgr2=0x557a43500680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a43500680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f796c0103f0 tx=0x7f796c0108b0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 -- 172.21.15.110:0/169537739 shutdown_connections 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a43508250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43505750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 --2- 172.21.15.110:0/169537739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a43500680 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 -- 172.21.15.110:0/169537739 >> 172.21.15.110:0/169537739 conn(0x557a434f74c0 msgr2=0x557a434f78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 -- 172.21.15.110:0/169537739 shutdown_connections 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 -- 172.21.15.110:0/169537739 wait complete. 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 -- start start 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.701+0000 7f797dbfd740 1 Processor -- start 2024-04-25T00:02:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43500260 0x557a4350a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43505dc0 0x557a43511c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f797dbfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557a43508d70 con 0x557a43500c50 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f797dbfd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557a4341bb50 con 0x557a43500260 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f797dbfd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557a43508ef0 con 0x557a43505dc0 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f797dbfd740 10 client.0 _ll_register_callbacks cb 0x557a43456ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49890/0 (socket says 172.21.15.110:49890) 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 -- 172.21.15.110:0/837737262 learned_addr learned my addr 172.21.15.110:0/837737262 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7978c93640 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43500260 0x557a4350a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7979494640 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43505dc0 0x557a43511c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 -- 172.21.15.110:0/837737262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43505dc0 msgr2=0x557a43511c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43505dc0 0x557a43511c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 -- 172.21.15.110:0/837737262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43500260 msgr2=0x557a4350a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43500260 0x557a4350a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 -- 172.21.15.110:0/837737262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a43240620 con 0x557a43500c50 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7979494640 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43505dc0 0x557a43511c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7973fff640 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350ac20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f79600bbe30 tx=0x7f79600f3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b5592c7f0ee8d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7953fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.702+0000 7f7953fff640 1 -- 172.21.15.110:0/837737262 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960049040 con 0x557a43500c50 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f7953fff640 1 -- 172.21.15.110:0/837737262 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f796003b460 con 0x557a43500c50 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 1 -- 172.21.15.110:0/837737262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f796c0113e0 con 0x557a43500c50 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f7953fff640 1 -- 172.21.15.110:0/837737262 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f79600044e0 con 0x557a43500c50 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 10 client.15327 fetch_fsmap learned FSMap version 24 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 1 -- 172.21.15.110:0/837737262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557a43514fd0 con 0x557a43500c50 2024-04-25T00:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f7953fff640 1 -- 172.21.15.110:0/837737262 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f796c0113e0 con 0x557a43500c50 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f7953fff640 1 -- 172.21.15.110:0/837737262 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f79600bc100 con 0x557a43500c50 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 10 client.15327 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 10 client.15327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 20 client.15327 populate_metadata read hostname 'smithi110' 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557a435166e0 0x557a43518b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 4 client.15327 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f797dbfd740 1 -- 172.21.15.110:0/837737262 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a4346d1f0 con 0x557a435166e0 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.703+0000 7f7978c93640 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557a435166e0 0x557a43518b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.704+0000 7f7953fff640 10 client.15327 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.704+0000 7f7978c93640 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557a435166e0 0x557a43518b80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e7f1ea83d79a8b server_cookie=15701db4919c4b51 in_seq=0 out_seq=0 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.704+0000 7f7953fff640 1 -- 172.21.15.110:0/837737262 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a4346d1f0 con 0x557a435166e0 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.704+0000 7f7953fff640 10 client.15327 handle_command_reply: tid=0 2024-04-25T00:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 client.15327 shutdown 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 -- 172.21.15.110:0/837737262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 msgr2=0x557a4350ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350ac20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f79600bbe30 tx=0x7f79600f3040 comp rx=0 tx=0).stop 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 -- 172.21.15.110:0/837737262 shutdown_connections 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557a435166e0 0x557a43518b80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43505dc0 0x557a43511c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f7953fff640 0 client.15327 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350ac20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 --2- 172.21.15.110:0/837737262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43500260 0x557a4350a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 -- 172.21.15.110:0/837737262 >> 172.21.15.110:0/837737262 conn(0x557a434f74c0 msgr2=0x557a434f65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 -- 172.21.15.110:0/837737262 shutdown_connections 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 1 -- 172.21.15.110:0/837737262 wait complete. 2024-04-25T00:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.705+0000 7f797dbfd740 20 client.15327 trim_cache size 0 max 16384 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f797dbfd740 1 Processor -- start 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f797dbfd740 1 -- start start 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a435092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43511c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a434f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f797dbfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a9070 con 0x557a43500260 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f797dbfd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a6940 con 0x557a43505dc0 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f797dbfd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a4790 con 0x557a43500c50 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7978c93640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a435092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7978c93640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a435092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49906/0 (socket says 172.21.15.110:49906) 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7978c93640 1 -- 172.21.15.110:0/2937294645 learned_addr learned my addr 172.21.15.110:0/2937294645 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7973fff640 1 --2- 172.21.15.110:0/2937294645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43511c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7978c93640 1 -- 172.21.15.110:0/2937294645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 msgr2=0x557a43511c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7978c93640 1 --2- 172.21.15.110:0/2937294645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43511c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7978c93640 1 -- 172.21.15.110:0/2937294645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 msgr2=0x557a434f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7978c93640 1 --2- 172.21.15.110:0/2937294645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a434f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.727+0000 7f7978c93640 1 -- 172.21.15.110:0/2937294645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4350af90 con 0x557a43500260 2024-04-25T00:02:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.728+0000 7f7978c93640 1 --2- 172.21.15.110:0/2937294645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a435092f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x557a435016c0 tx=0x7f79680045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb178b09a0fe8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.728+0000 7f795e7fc640 1 -- 172.21.15.110:0/2937294645 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796801f050 con 0x557a43500260 2024-04-25T00:02:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.728+0000 7f795e7fc640 1 -- 172.21.15.110:0/2937294645 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f796801f6a0 con 0x557a43500260 2024-04-25T00:02:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.728+0000 7f795e7fc640 1 -- 172.21.15.110:0/2937294645 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968010c20 con 0x557a43500260 2024-04-25T00:02:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.728+0000 7f797dbfd740 1 -- 172.21.15.110:0/2937294645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 msgr2=0x557a435092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.728+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2937294645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a435092f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x557a435016c0 tx=0x7f79680045d0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 -- 172.21.15.110:0/2937294645 shutdown_connections 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2937294645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a434f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2937294645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500c50 0x557a43511c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2937294645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500260 0x557a435092f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 -- 172.21.15.110:0/2937294645 >> 172.21.15.110:0/2937294645 conn(0x557a434f74c0 msgr2=0x557a434f78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 -- 172.21.15.110:0/2937294645 shutdown_connections 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 -- 172.21.15.110:0/2937294645 wait complete. 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 -- start start 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 Processor -- start 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500260 0x557a4350ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a435104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a4e00 con 0x557a43500c50 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a6270 con 0x557a43505dc0 2024-04-25T00:02:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79741a54d0 con 0x557a43500260 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f797dbfd740 10 client.0 _ll_register_callbacks cb 0x7f7960031980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.729+0000 7f7973fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7973fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49916/0 (socket says 172.21.15.110:49916) 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7973fff640 1 -- 172.21.15.110:0/2803136609 learned_addr learned my addr 172.21.15.110:0/2803136609 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7979494640 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a435104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7978c93640 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500260 0x557a4350ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7973fff640 1 -- 172.21.15.110:0/2803136609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500260 msgr2=0x557a4350ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7973fff640 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500260 0x557a4350ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7973fff640 1 -- 172.21.15.110:0/2803136609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 msgr2=0x557a435104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7973fff640 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a435104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7973fff640 1 -- 172.21.15.110:0/2803136609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a43510aa0 con 0x557a43500c50 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7973fff640 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350f4a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79600c45e0 tx=0x7f79600bc4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7d3f86b3b7856f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7953fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:02:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7953fff640 1 -- 172.21.15.110:0/2803136609 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960049070 con 0x557a43500c50 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7953fff640 1 -- 172.21.15.110:0/2803136609 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f79600efa70 con 0x557a43500c50 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f797dbfd740 1 -- 172.21.15.110:0/2803136609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f796801f6a0 con 0x557a43500c50 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7953fff640 1 -- 172.21.15.110:0/2803136609 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f79600d8070 con 0x557a43500c50 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f797dbfd740 10 client.15333 fetch_fsmap learned FSMap version 24 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f797dbfd740 1 -- 172.21.15.110:0/2803136609 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557a43511e20 con 0x557a43500c50 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.730+0000 7f7953fff640 1 -- 172.21.15.110:0/2803136609 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f796801f6a0 con 0x557a43500c50 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f7953fff640 1 -- 172.21.15.110:0/2803136609 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f796003bbb0 con 0x557a43500c50 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f797dbfd740 10 client.15333 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f797dbfd740 10 client.15333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f797dbfd740 20 client.15333 populate_metadata read hostname 'smithi110' 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557a435166e0 0x557a4357f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f797dbfd740 4 client.15333 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f797dbfd740 1 -- 172.21.15.110:0/2803136609 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a43467540 con 0x557a435166e0 2024-04-25T00:02:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f7978c93640 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557a435166e0 0x557a4357f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f7978c93640 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557a435166e0 0x557a4357f2a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5729b1778809231 server_cookie=573c5c571ab4963a in_seq=0 out_seq=0 2024-04-25T00:02:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.731+0000 7f7953fff640 10 client.15333 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f7953fff640 1 -- 172.21.15.110:0/2803136609 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a43467540 con 0x557a435166e0 2024-04-25T00:02:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f7953fff640 10 client.15333 handle_command_reply: tid=0 2024-04-25T00:02:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f797dbfd740 1 client.15333 shutdown 2024-04-25T00:02:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f797dbfd740 1 -- 172.21.15.110:0/2803136609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 msgr2=0x557a4350f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350f4a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79600c45e0 tx=0x7f79600bc4c0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f797dbfd740 1 -- 172.21.15.110:0/2803136609 shutdown_connections 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557a435166e0 0x557a4357f2a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a43505dc0 0x557a435104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a43500c50 0x557a4350f4a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f7953fff640 0 client.15333 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.746+0000 7f797dbfd740 1 --2- 172.21.15.110:0/2803136609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a43500260 0x557a4350ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797dbfd740 1 -- 172.21.15.110:0/2803136609 >> 172.21.15.110:0/2803136609 conn(0x557a434f74c0 msgr2=0x557a434f78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797dbfd740 1 -- 172.21.15.110:0/2803136609 shutdown_connections 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797dbfd740 1 -- 172.21.15.110:0/2803136609 wait complete. 2024-04-25T00:02:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797dbfd740 20 client.15333 trim_cache size 0 max 16384 2024-04-25T00:02:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:02:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:02:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:02:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:02:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 -- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79640a2c30 msgr2=0x7f79640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79640a2c30 0x7f79640a50f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f796c020000 tx=0x7f796c01d000 comp rx=0 tx=0).stop 2024-04-25T00:02:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 -- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 msgr2=0x7f797419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 0x7f797419c700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7960005940 tx=0x7f79600042f0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 -- 172.21.15.110:0/1213810950 shutdown_connections 2024-04-25T00:02:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79640a2c30 0x7f79640a50f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7974102250 0x7f797419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7974101890 0x7f797419c700 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 --2- 172.21.15.110:0/1213810950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79741006a0 0x7f7974069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.747+0000 7f797aef7640 1 -- 172.21.15.110:0/1213810950 >> 172.21.15.110:0/1213810950 conn(0x7f79740fbc10 msgr2=0x7f7974104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.748+0000 7f797aef7640 1 -- 172.21.15.110:0/1213810950 shutdown_connections 2024-04-25T00:02:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:46.748+0000 7f797aef7640 1 -- 172.21.15.110:0/1213810950 wait complete. 2024-04-25T00:02:47.261 INFO:teuthology.orchestra.run.smithi110.stderr:2600000 of 3200000 tuples (81%) done (elapsed 24.48 s, remaining 5.65 s) 2024-04-25T00:02:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:47 smithi173 ceph-mon[32402]: pgmap v373: 97 pgs: 97 active+clean; 528 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 43 op/s 2024-04-25T00:02:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:47 smithi188 ceph-mon[33285]: pgmap v373: 97 pgs: 97 active+clean; 528 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 43 op/s 2024-04-25T00:02:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:47 smithi110 ceph-mon[29187]: pgmap v373: 97 pgs: 97 active+clean; 528 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 43 op/s 2024-04-25T00:02:48.332 INFO:teuthology.orchestra.run.smithi110.stderr:2700000 of 3200000 tuples (84%) done (elapsed 25.55 s, remaining 4.73 s) 2024-04-25T00:02:49.102 INFO:teuthology.orchestra.run.smithi110.stderr:2800000 of 3200000 tuples (87%) done (elapsed 26.32 s, remaining 3.76 s) 2024-04-25T00:02:49.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:49 smithi173 ceph-mon[32402]: pgmap v374: 97 pgs: 97 active+clean; 604 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 MiB/s wr, 51 op/s 2024-04-25T00:02:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:49 smithi188 ceph-mon[33285]: pgmap v374: 97 pgs: 97 active+clean; 604 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 MiB/s wr, 51 op/s 2024-04-25T00:02:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:49 smithi110 ceph-mon[29187]: pgmap v374: 97 pgs: 97 active+clean; 604 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 MiB/s wr, 51 op/s 2024-04-25T00:02:50.229 INFO:teuthology.orchestra.run.smithi110.stderr:2900000 of 3200000 tuples (90%) done (elapsed 27.45 s, remaining 2.84 s) 2024-04-25T00:02:51.255 INFO:teuthology.orchestra.run.smithi110.stderr:3000000 of 3200000 tuples (93%) done (elapsed 28.47 s, remaining 1.90 s) 2024-04-25T00:02:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:51 smithi173 ceph-mon[32402]: pgmap v375: 97 pgs: 97 active+clean; 634 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 MiB/s wr, 41 op/s 2024-04-25T00:02:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:51 smithi188 ceph-mon[33285]: pgmap v375: 97 pgs: 97 active+clean; 634 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 MiB/s wr, 41 op/s 2024-04-25T00:02:52.032 INFO:teuthology.orchestra.run.smithi110.stderr:3100000 of 3200000 tuples (96%) done (elapsed 29.25 s, remaining 0.94 s) 2024-04-25T00:02:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:51 smithi110 ceph-mon[29187]: pgmap v375: 97 pgs: 97 active+clean; 634 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 MiB/s wr, 41 op/s 2024-04-25T00:02:53.138 INFO:teuthology.orchestra.run.smithi110.stderr:3200000 of 3200000 tuples (100%) done (elapsed 30.35 s, remaining 0.00 s) 2024-04-25T00:02:53.214 INFO:teuthology.orchestra.run.smithi110.stderr:vacuuming... 2024-04-25T00:02:53.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:53 smithi173 ceph-mon[32402]: pgmap v376: 97 pgs: 97 active+clean; 670 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 43 op/s 2024-04-25T00:02:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:53 smithi188 ceph-mon[33285]: pgmap v376: 97 pgs: 97 active+clean; 670 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 43 op/s 2024-04-25T00:02:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:53 smithi110 ceph-mon[29187]: pgmap v376: 97 pgs: 97 active+clean; 670 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 43 op/s 2024-04-25T00:02:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:55 smithi188 ceph-mon[33285]: pgmap v377: 97 pgs: 97 active+clean; 748 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 MiB/s wr, 46 op/s 2024-04-25T00:02:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:55 smithi110 ceph-mon[29187]: pgmap v377: 97 pgs: 97 active+clean; 748 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 MiB/s wr, 46 op/s 2024-04-25T00:02:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:55 smithi173 ceph-mon[32402]: pgmap v377: 97 pgs: 97 active+clean; 748 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 MiB/s wr, 46 op/s 2024-04-25T00:02:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:57 smithi188 ceph-mon[33285]: pgmap v378: 97 pgs: 97 active+clean; 758 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 44 MiB/s wr, 39 op/s 2024-04-25T00:02:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:57 smithi110 ceph-mon[29187]: pgmap v378: 97 pgs: 97 active+clean; 758 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 44 MiB/s wr, 39 op/s 2024-04-25T00:02:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:57 smithi173 ceph-mon[32402]: pgmap v378: 97 pgs: 97 active+clean; 758 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 44 MiB/s wr, 39 op/s 2024-04-25T00:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.850+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/3615536259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc107500 msgr2=0x7fe7cc109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.850+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/3615536259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc107500 0x7fe7cc109910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009ca0 tx=0x7fe7c402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/3615536259 shutdown_connections 2024-04-25T00:02:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/3615536259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc107500 0x7fe7cc109910 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/3615536259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7cc104bd0 0x7fe7cc106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/3615536259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7cc1022a0 0x7fe7cc104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/3615536259 >> 172.21.15.110:0/3615536259 conn(0x7fe7cc0fbc50 msgr2=0x7fe7cc0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/3615536259 shutdown_connections 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/3615536259 wait complete. 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 Processor -- start 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 -- start start 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7cc1022a0 0x7fe7cc197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc104bd0 0x7fe7cc197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7cc107500 0x7fe7cc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10d310 con 0x7fe7cc104bd0 2024-04-25T00:02:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10d190 con 0x7fe7cc1022a0 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.851+0000 7fe7d2bc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc10d490 con 0x7fe7cc107500 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc104bd0 0x7fe7cc197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc104bd0 0x7fe7cc197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40882/0 (socket says 172.21.15.110:40882) 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 -- 172.21.15.110:0/1029612298 learned_addr learned my addr 172.21.15.110:0/1029612298 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7d1161640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7cc107500 0x7fe7cc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7d0960640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7cc1022a0 0x7fe7cc197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 -- 172.21.15.110:0/1029612298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7cc107500 msgr2=0x7fe7cc19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7cc107500 0x7fe7cc19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 -- 172.21.15.110:0/1029612298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7cc1022a0 msgr2=0x7fe7cc197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7cc1022a0 0x7fe7cc197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 -- 172.21.15.110:0/1029612298 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc19f500 con 0x7fe7cc104bd0 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc104bd0 0x7fe7cc197e70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e9a0 tx=0x7fe7b800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7c9ffb640 1 -- 172.21.15.110:0/1029612298 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800cd80 con 0x7fe7cc104bd0 2024-04-25T00:02:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7c9ffb640 1 -- 172.21.15.110:0/1029612298 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7b80035f0 con 0x7fe7cc104bd0 2024-04-25T00:02:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/1029612298 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc19f7f0 con 0x7fe7cc104bd0 2024-04-25T00:02:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.852+0000 7fe7c9ffb640 1 -- 172.21.15.110:0/1029612298 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8010690 con 0x7fe7cc104bd0 2024-04-25T00:02:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.853+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/1029612298 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc19fd90 con 0x7fe7cc104bd0 2024-04-25T00:02:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7c9ffb640 1 -- 172.21.15.110:0/1029612298 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe7b80037a0 con 0x7fe7cc104bd0 2024-04-25T00:02:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7d59a8740 1 Processor -- start 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7d59a8740 1 -- start start 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 0x5598f51c2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5598f5120cb0 con 0x5598f51c29b0 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5598f50c95b0 con 0x5598f51bd840 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5598f511cc70 con 0x5598f51bce50 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7c9ffb640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe7bc0a2cb0 0x7fe7bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.855+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40890/0 (socket says 172.21.15.110:40890) 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7cbfff640 1 -- 172.21.15.110:0/3874362026 learned_addr learned my addr 172.21.15.110:0/3874362026 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7d1161640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe7bc0a2cb0 0x7fe7bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7c9ffb640 1 -- 172.21.15.110:0/1029612298 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe7b80cf2c0 con 0x7fe7cc104bd0 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7d0960640 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 0x5598f51c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7d1161640 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7cbfff640 1 -- 172.21.15.110:0/3874362026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 msgr2=0x5598f51bd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7cbfff640 1 -- 172.21.15.110:0/3874362026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 msgr2=0x5598f51c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 0x5598f51c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7cbfff640 1 -- 172.21.15.110:0/3874362026 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f4e35210 con 0x5598f51c29b0 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7d1161640 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51c4e40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe7b809bae0 tx=0x7fe7b806ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc85df5380238e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7d1161640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe7bc0a2cb0 0x7fe7bc0a5170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5598f51be0a0 tx=0x7fe7c4005df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7b6ffd640 1 -- 172.21.15.110:0/3874362026 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8003b10 con 0x5598f51c29b0 2024-04-25T00:02:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7b6ffd640 1 -- 172.21.15.110:0/3874362026 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7b8066c30 con 0x5598f51c29b0 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.856+0000 7fe7b6ffd640 1 -- 172.21.15.110:0/3874362026 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80981c0 con 0x5598f51c29b0 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 -- 172.21.15.110:0/3874362026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 msgr2=0x5598f51c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51c4e40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe7b809bae0 tx=0x7fe7b806ec10 comp rx=0 tx=0).stop 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 -- 172.21.15.110:0/3874362026 shutdown_connections 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51c4e40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 0x5598f51c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/3874362026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 -- 172.21.15.110:0/3874362026 >> 172.21.15.110:0/3874362026 conn(0x5598f51b84c0 msgr2=0x5598f51b88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 -- 172.21.15.110:0/3874362026 shutdown_connections 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 -- 172.21.15.110:0/3874362026 wait complete. 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.857+0000 7fe7d59a8740 1 -- start start 2024-04-25T00:02:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d59a8740 1 Processor -- start 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bce50 0x5598f51cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51c29b0 0x5598f51d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5598f51c9c20 con 0x5598f51bd840 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5598f50dcb50 con 0x5598f51bce50 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5598f51c9da0 con 0x5598f51c29b0 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d59a8740 10 client.0 _ll_register_callbacks cb 0x5598f5117ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40900/0 (socket says 172.21.15.110:40900) 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7cbfff640 1 -- 172.21.15.110:0/962105043 learned_addr learned my addr 172.21.15.110:0/962105043 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d1161640 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51c29b0 0x5598f51d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d0960640 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bce50 0x5598f51cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7cbfff640 1 -- 172.21.15.110:0/962105043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51c29b0 msgr2=0x5598f51d2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51c29b0 0x5598f51d2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7cbfff640 1 -- 172.21.15.110:0/962105043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bce50 msgr2=0x5598f51cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bce50 0x5598f51cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7cbfff640 1 -- 172.21.15.110:0/962105043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f4f01620 con 0x5598f51bd840 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.858+0000 7fe7d1161640 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51c29b0 0x5598f51d2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51cbb60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80a00c0 tx=0x7fe7b809ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4761d793b39a1af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7abfff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7abfff640 1 -- 172.21.15.110:0/962105043 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b809bb50 con 0x5598f51bd840 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7abfff640 1 -- 172.21.15.110:0/962105043 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe7b80a4540 con 0x5598f51bd840 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 1 -- 172.21.15.110:0/962105043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b8066c30 con 0x5598f51bd840 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7abfff640 1 -- 172.21.15.110:0/962105043 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe7b809c250 con 0x5598f51bd840 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 10 client.15345 fetch_fsmap learned FSMap version 24 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 1 -- 172.21.15.110:0/962105043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5598f51d5f10 con 0x5598f51bd840 2024-04-25T00:02:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7abfff640 1 -- 172.21.15.110:0/962105043 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7b8066c30 con 0x5598f51bd840 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7abfff640 1 -- 172.21.15.110:0/962105043 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe7b8097aa0 con 0x5598f51bd840 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 10 client.15345 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 20 client.15345 populate_metadata read hostname 'smithi110' 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5598f51d7260 0x5598f51d9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 4 client.15345 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.859+0000 7fe7d59a8740 1 -- 172.21.15.110:0/962105043 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598f512e1f0 con 0x5598f51d7260 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.860+0000 7fe7d0960640 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5598f51d7260 0x5598f51d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.860+0000 7fe7abfff640 10 client.15345 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.860+0000 7fe7d0960640 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5598f51d7260 0x5598f51d9650 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de999ce6f3cea97 server_cookie=a5bf86c7eb580505 in_seq=0 out_seq=0 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7abfff640 1 -- 172.21.15.110:0/962105043 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598f512e1f0 con 0x5598f51d7260 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7abfff640 10 client.15345 handle_command_reply: tid=0 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 client.15345 shutdown 2024-04-25T00:02:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 -- 172.21.15.110:0/962105043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 msgr2=0x5598f51cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51cbb60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80a00c0 tx=0x7fe7b809ff00 comp rx=0 tx=0).stop 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 -- 172.21.15.110:0/962105043 shutdown_connections 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5598f51d7260 0x5598f51d9650 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51c29b0 0x5598f51d2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51cbb60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7abfff640 0 client.15345 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/962105043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bce50 0x5598f51cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 -- 172.21.15.110:0/962105043 >> 172.21.15.110:0/962105043 conn(0x5598f51b84c0 msgr2=0x5598f51b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 -- 172.21.15.110:0/962105043 shutdown_connections 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 1 -- 172.21.15.110:0/962105043 wait complete. 2024-04-25T00:02:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.861+0000 7fe7d59a8740 20 client.15345 trim_cache size 0 max 16384 2024-04-25T00:02:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d59a8740 1 Processor -- start 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d59a8740 1 -- start start 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51c29b0 0x5598f51d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1a9120 con 0x5598f51bd840 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1a69f0 con 0x5598f51c29b0 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1a4840 con 0x5598f51bce50 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51bc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40906/0 (socket says 172.21.15.110:40906) 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d0960640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7cbfff640 1 -- 172.21.15.110:0/431715940 learned_addr learned my addr 172.21.15.110:0/431715940 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d1161640 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51c29b0 0x5598f51d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d0960640 1 -- 172.21.15.110:0/431715940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51c29b0 msgr2=0x5598f51d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d0960640 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51c29b0 0x5598f51d2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d0960640 1 -- 172.21.15.110:0/431715940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 msgr2=0x5598f51bc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d0960640 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51bc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7d0960640 1 -- 172.21.15.110:0/431715940 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f51d99c0 con 0x5598f51bce50 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.876+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51bc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.877+0000 7fe7d0960640 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bbb50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5598f51d9e30 tx=0x7fe7c0014840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9df3fd13030a4e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.877+0000 7fe7b67fc640 1 -- 172.21.15.110:0/431715940 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0024070 con 0x5598f51bce50 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.877+0000 7fe7b67fc640 1 -- 172.21.15.110:0/431715940 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7c001e470 con 0x5598f51bce50 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.877+0000 7fe7b67fc640 1 -- 172.21.15.110:0/431715940 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00039e0 con 0x5598f51bce50 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 -- 172.21.15.110:0/431715940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 msgr2=0x5598f51bbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bbb50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5598f51d9e30 tx=0x7fe7c0014840 comp rx=0 tx=0).stop 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 -- 172.21.15.110:0/431715940 shutdown_connections 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51c29b0 0x5598f51d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51bd840 0x5598f51bc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/431715940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51bbb50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 -- 172.21.15.110:0/431715940 >> 172.21.15.110:0/431715940 conn(0x5598f51b84c0 msgr2=0x5598f51bb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 -- 172.21.15.110:0/431715940 shutdown_connections 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 -- 172.21.15.110:0/431715940 wait complete. 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 -- start start 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 Processor -- start 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 0x5598f51d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.878+0000 7fe7d59a8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1a4eb0 con 0x5598f51c29b0 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1a6320 con 0x5598f51bd840 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d59a8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1a5580 con 0x5598f51bce50 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d59a8740 10 client.0 _ll_register_callbacks cb 0x7fe7b8009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d1161640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:47314/0 (socket says 172.21.15.110:47314) 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 -- 172.21.15.110:0/1082319691 learned_addr learned my addr 172.21.15.110:0/1082319691 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 0x5598f51d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 -- 172.21.15.110:0/1082319691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 msgr2=0x5598f51d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 0x5598f51d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 -- 172.21.15.110:0/1082319691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 msgr2=0x5598f51d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 -- 172.21.15.110:0/1082319691 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f51d1850 con 0x5598f51bce50 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d0960640 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51cfd10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0000c00 tx=0x7fe7c0014190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b33fb755eb3ec5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7abfff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7abfff640 1 -- 172.21.15.110:0/1082319691 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c002d030 con 0x5598f51bce50 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7abfff640 1 -- 172.21.15.110:0/1082319691 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe7c001e620 con 0x5598f51bce50 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d59a8740 1 -- 172.21.15.110:0/1082319691 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c001e470 con 0x5598f51bce50 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7abfff640 1 -- 172.21.15.110:0/1082319691 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe7c0024040 con 0x5598f51bce50 2024-04-25T00:02:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.879+0000 7fe7d59a8740 10 client.24907 fetch_fsmap learned FSMap version 24 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7d59a8740 1 -- 172.21.15.110:0/1082319691 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5598f51d2bd0 con 0x5598f51bce50 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7abfff640 1 -- 172.21.15.110:0/1082319691 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7c001e470 con 0x5598f51bce50 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7abfff640 1 -- 172.21.15.110:0/1082319691 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe7c0003760 con 0x5598f51bce50 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7d59a8740 10 client.24907 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7d59a8740 10 client.24907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7d59a8740 20 client.24907 populate_metadata read hostname 'smithi110' 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5598f523dc90 0x5598f52400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7d59a8740 4 client.24907 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7d59a8740 1 -- 172.21.15.110:0/1082319691 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598f5128540 con 0x5598f523dc90 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5598f523dc90 0x5598f52400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7abfff640 10 client.24907 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.880+0000 7fe7cbfff640 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5598f523dc90 0x5598f52400b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350fe9967b22847 server_cookie=c6fbcfa4e00e8fa5 in_seq=0 out_seq=0 2024-04-25T00:02:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.895+0000 7fe7abfff640 1 -- 172.21.15.110:0/1082319691 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598f5128540 con 0x5598f523dc90 2024-04-25T00:02:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.895+0000 7fe7abfff640 10 client.24907 handle_command_reply: tid=0 2024-04-25T00:02:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.895+0000 7fe7d59a8740 1 client.24907 shutdown 2024-04-25T00:02:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.895+0000 7fe7d59a8740 1 -- 172.21.15.110:0/1082319691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 msgr2=0x5598f51cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.895+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51cfd10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0000c00 tx=0x7fe7c0014190 comp rx=0 tx=0).stop 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 1 -- 172.21.15.110:0/1082319691 shutdown_connections 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5598f523dc90 0x5598f52400b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598f51c29b0 0x5598f51d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5598f51bd840 0x5598f51d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7abfff640 0 client.24907 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 1 --2- 172.21.15.110:0/1082319691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5598f51bce50 0x5598f51cfd10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 1 -- 172.21.15.110:0/1082319691 >> 172.21.15.110:0/1082319691 conn(0x5598f51b84c0 msgr2=0x5598f51bbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 1 -- 172.21.15.110:0/1082319691 shutdown_connections 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 1 -- 172.21.15.110:0/1082319691 wait complete. 2024-04-25T00:02:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d59a8740 20 client.24907 trim_cache size 0 max 16384 2024-04-25T00:02:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:02:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:02:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:02:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe7bc0a2cb0 msgr2=0x7fe7bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe7bc0a2cb0 0x7fe7bc0a5170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5598f51be0a0 tx=0x7fe7c4005df0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc104bd0 msgr2=0x7fe7cc197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.896+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc104bd0 0x7fe7cc197e70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e9a0 tx=0x7fe7b800ee60 comp rx=0 tx=0).stop 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.897+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/1029612298 shutdown_connections 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.897+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe7bc0a2cb0 0x7fe7bc0a5170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.897+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7cc107500 0x7fe7cc19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.897+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7cc104bd0 0x7fe7cc197e70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.897+0000 7fe7d2bc4640 1 --2- 172.21.15.110:0/1029612298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7cc1022a0 0x7fe7cc197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.897+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/1029612298 >> 172.21.15.110:0/1029612298 conn(0x7fe7cc0fbc50 msgr2=0x7fe7cc102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:02:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.897+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/1029612298 shutdown_connections 2024-04-25T00:02:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:02:59.897+0000 7fe7d2bc4640 1 -- 172.21.15.110:0/1029612298 wait complete. 2024-04-25T00:03:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:02:59 smithi188 ceph-mon[33285]: pgmap v379: 97 pgs: 97 active+clean; 772 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 55 MiB/s wr, 49 op/s 2024-04-25T00:03:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:02:59 smithi110 ceph-mon[29187]: pgmap v379: 97 pgs: 97 active+clean; 772 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 55 MiB/s wr, 49 op/s 2024-04-25T00:03:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:02:59 smithi173 ceph-mon[32402]: pgmap v379: 97 pgs: 97 active+clean; 772 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 55 MiB/s wr, 49 op/s 2024-04-25T00:03:00.371 INFO:teuthology.orchestra.run.smithi110.stderr:creating primary keys... 2024-04-25T00:03:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:01 smithi188 ceph-mon[33285]: pgmap v380: 97 pgs: 97 active+clean; 772 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 MiB/s wr, 41 op/s 2024-04-25T00:03:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:01 smithi110 ceph-mon[29187]: pgmap v380: 97 pgs: 97 active+clean; 772 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 MiB/s wr, 41 op/s 2024-04-25T00:03:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:01 smithi173 ceph-mon[32402]: pgmap v380: 97 pgs: 97 active+clean; 772 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 MiB/s wr, 41 op/s 2024-04-25T00:03:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:03 smithi188 ceph-mon[33285]: pgmap v381: 97 pgs: 97 active+clean; 798 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 MiB/s wr, 43 op/s 2024-04-25T00:03:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:03 smithi110 ceph-mon[29187]: pgmap v381: 97 pgs: 97 active+clean; 798 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 MiB/s wr, 43 op/s 2024-04-25T00:03:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:03 smithi173 ceph-mon[32402]: pgmap v381: 97 pgs: 97 active+clean; 798 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 MiB/s wr, 43 op/s 2024-04-25T00:03:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:05 smithi188 ceph-mon[33285]: pgmap v382: 97 pgs: 97 active+clean; 842 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 59 MiB/s wr, 50 op/s 2024-04-25T00:03:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:05 smithi110 ceph-mon[29187]: pgmap v382: 97 pgs: 97 active+clean; 842 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 59 MiB/s wr, 50 op/s 2024-04-25T00:03:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:05 smithi173 ceph-mon[32402]: pgmap v382: 97 pgs: 97 active+clean; 842 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 59 MiB/s wr, 50 op/s 2024-04-25T00:03:06.429 INFO:teuthology.orchestra.run.smithi110.stderr:done in 43.72 s (drop tables 0.01 s, create tables 0.05 s, client-side generate 30.45 s, vacuum 7.16 s, primary keys 6.06 s). 2024-04-25T00:03:06.433 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-04-25T00:03:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:starting vacuum...end. 2024-04-25T00:03:07.904 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:07 smithi173 ceph-mon[32402]: pgmap v383: 97 pgs: 97 active+clean; 858 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 MiB/s wr, 43 op/s 2024-04-25T00:03:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:07 smithi188 ceph-mon[33285]: pgmap v383: 97 pgs: 97 active+clean; 858 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 MiB/s wr, 43 op/s 2024-04-25T00:03:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:07 smithi110 ceph-mon[29187]: pgmap v383: 97 pgs: 97 active+clean; 858 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 MiB/s wr, 43 op/s 2024-04-25T00:03:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:09 smithi188 ceph-mon[33285]: pgmap v384: 97 pgs: 97 active+clean; 943 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 58 MiB/s wr, 69 op/s 2024-04-25T00:03:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:09 smithi110 ceph-mon[29187]: pgmap v384: 97 pgs: 97 active+clean; 943 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 58 MiB/s wr, 69 op/s 2024-04-25T00:03:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:09 smithi173 ceph-mon[32402]: pgmap v384: 97 pgs: 97 active+clean; 943 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 58 MiB/s wr, 69 op/s 2024-04-25T00:03:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:10 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:11.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:10 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:11.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:10 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 5.0 s, 1973.8 tps, lat 47.704 ms stddev 52.543 2024-04-25T00:03:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:11 smithi188 ceph-mon[33285]: pgmap v385: 97 pgs: 97 active+clean; 967 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 47 MiB/s wr, 65 op/s 2024-04-25T00:03:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:11 smithi110 ceph-mon[29187]: pgmap v385: 97 pgs: 97 active+clean; 967 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 47 MiB/s wr, 65 op/s 2024-04-25T00:03:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:11 smithi173 ceph-mon[32402]: pgmap v385: 97 pgs: 97 active+clean; 967 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 47 MiB/s wr, 65 op/s 2024-04-25T00:03:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 -- 172.21.15.110:0/3840463737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c101880 msgr2=0x7f072c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 --2- 172.21.15.110:0/3840463737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c101880 0x7f072c101d00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f071400b9f0 tx=0x7f071402c0a0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 -- 172.21.15.110:0/3840463737 shutdown_connections 2024-04-25T00:03:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 --2- 172.21.15.110:0/3840463737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c102240 0x7f072c108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 --2- 172.21.15.110:0/3840463737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c101880 0x7f072c101d00 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f071400b9f0 tx=0x7f071402c0a0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 --2- 172.21.15.110:0/3840463737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f072c100690 0x7f072c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 -- 172.21.15.110:0/3840463737 >> 172.21.15.110:0/3840463737 conn(0x7f072c0fbc00 msgr2=0x7f072c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 -- 172.21.15.110:0/3840463737 shutdown_connections 2024-04-25T00:03:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.034+0000 7f0732ca5640 1 -- 172.21.15.110:0/3840463737 wait complete. 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0732ca5640 1 Processor -- start 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0732ca5640 1 -- start start 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0732ca5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f072c100690 0x7f072c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0732ca5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c102240 0x7f072c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0732ca5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c1a3620 0x7f072c1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0732ca5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10b0c0 con 0x7f072c102240 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0732ca5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10af40 con 0x7f072c100690 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0732ca5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10b240 con 0x7f072c1a3620 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0731242640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c1a3620 0x7f072c1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0731242640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c1a3620 0x7f072c1a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:56912/0 (socket says 172.21.15.110:56912) 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0731242640 1 -- 172.21.15.110:0/2329996350 learned_addr learned my addr 172.21.15.110:0/2329996350 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f072bfff640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c102240 0x7f072c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.035+0000 7f0730a41640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f072c100690 0x7f072c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.036+0000 7f0731242640 1 -- 172.21.15.110:0/2329996350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f072c100690 msgr2=0x7f072c19e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.036+0000 7f0731242640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f072c100690 0x7f072c19e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.036+0000 7f0731242640 1 -- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c102240 msgr2=0x7f072c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.036+0000 7f0731242640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c102240 0x7f072c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.036+0000 7f0731242640 1 -- 172.21.15.110:0/2329996350 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c1a6080 con 0x7f072c1a3620 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.036+0000 7f072bfff640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c102240 0x7f072c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.036+0000 7f0731242640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c1a3620 0x7f072c1a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f071c00efc0 tx=0x7f071c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.036+0000 7f0729ffb640 1 -- 172.21.15.110:0/2329996350 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c003970 con 0x7f072c1a3620 2024-04-25T00:03:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.037+0000 7f0729ffb640 1 -- 172.21.15.110:0/2329996350 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f071c004020 con 0x7f072c1a3620 2024-04-25T00:03:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.037+0000 7f0729ffb640 1 -- 172.21.15.110:0/2329996350 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c00b920 con 0x7f072c1a3620 2024-04-25T00:03:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.037+0000 7f0732ca5640 1 -- 172.21.15.110:0/2329996350 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072c1a6370 con 0x7f072c1a3620 2024-04-25T00:03:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.038+0000 7f0732ca5640 1 -- 172.21.15.110:0/2329996350 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072c1a6840 con 0x7f072c1a3620 2024-04-25T00:03:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.041+0000 7f0729ffb640 1 -- 172.21.15.110:0/2329996350 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f071c01d020 con 0x7f072c1a3620 2024-04-25T00:03:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.041+0000 7f0729ffb640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f07180a2cb0 0x7f07180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f07359ba740 1 Processor -- start 2024-04-25T00:03:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f07359ba740 1 -- start start 2024-04-25T00:03:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f0729ffb640 1 -- 172.21.15.110:0/2329996350 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f071c0cee20 con 0x7f072c1a3620 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f07359ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae72270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f07359ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae72840 0x55befae77340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f0730a41640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f07180a2cb0 0x7f07180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f07359ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae779b0 0x55befae79e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f07359ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55befadd5cb0 con 0x55befae72840 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f07359ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55befad7e5b0 con 0x55befae779b0 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f07359ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55befadd1c70 con 0x55befae71e50 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f072bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae72840 0x55befae77340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f072bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae72840 0x55befae77340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35850/0 (socket says 172.21.15.110:35850) 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f072bfff640 1 -- 172.21.15.110:0/1896949330 learned_addr learned my addr 172.21.15.110:0/1896949330 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.042+0000 7f0731242640 1 --2- 172.21.15.110:0/1896949330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae779b0 0x55befae79e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0731242640 1 -- 172.21.15.110:0/1896949330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 msgr2=0x55befae72270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0731242640 1 --2- 172.21.15.110:0/1896949330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae72270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0731242640 1 -- 172.21.15.110:0/1896949330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae72840 msgr2=0x55befae77340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0731242640 1 --2- 172.21.15.110:0/1896949330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae72840 0x55befae77340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0730a41640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f07180a2cb0 0x7f07180a5170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f072c1016e0 tx=0x7f0724005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0731242640 1 -- 172.21.15.110:0/1896949330 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55befaaea210 con 0x55befae779b0 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0731242640 1 --2- 172.21.15.110:0/1896949330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae779b0 0x55befae79e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f071c00cd00 tx=0x7f071c09b070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364ba89f69291799 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0722ffd640 1 -- 172.21.15.110:0/1896949330 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c004540 con 0x55befae779b0 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.043+0000 7f0722ffd640 1 -- 172.21.15.110:0/1896949330 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f071c086070 con 0x55befae779b0 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.044+0000 7f0722ffd640 1 -- 172.21.15.110:0/1896949330 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c0a42c0 con 0x55befae779b0 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.044+0000 7f07359ba740 1 -- 172.21.15.110:0/1896949330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae779b0 msgr2=0x55befae79e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.044+0000 7f07359ba740 1 --2- 172.21.15.110:0/1896949330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae779b0 0x55befae79e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f071c00cd00 tx=0x7f071c09b070 comp rx=0 tx=0).stop 2024-04-25T00:03:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 -- 172.21.15.110:0/1896949330 shutdown_connections 2024-04-25T00:03:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 --2- 172.21.15.110:0/1896949330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae779b0 0x55befae79e40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 --2- 172.21.15.110:0/1896949330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae72840 0x55befae77340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 --2- 172.21.15.110:0/1896949330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae72270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 -- 172.21.15.110:0/1896949330 >> 172.21.15.110:0/1896949330 conn(0x55befae6d4c0 msgr2=0x55befae6d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 -- 172.21.15.110:0/1896949330 shutdown_connections 2024-04-25T00:03:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 -- 172.21.15.110:0/1896949330 wait complete. 2024-04-25T00:03:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 -- start start 2024-04-25T00:03:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.045+0000 7f07359ba740 1 Processor -- start 2024-04-25T00:03:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f07359ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae7c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f07359ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae7c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f07359ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f07359ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55befae7a960 con 0x55befae779b0 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f07359ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55befad91b50 con 0x55befae72840 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f0730a41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae7c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f0730a41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae7c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:56926/0 (socket says 172.21.15.110:56926) 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f0730a41640 1 -- 172.21.15.110:0/271118072 learned_addr learned my addr 172.21.15.110:0/271118072 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55befae7aae0 con 0x55befae71e50 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f0731242640 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f072bfff640 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae7c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.046+0000 7f07359ba740 10 client.0 _ll_register_callbacks cb 0x55befadccca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f0730a41640 1 -- 172.21.15.110:0/271118072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 msgr2=0x55befae7c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f0730a41640 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae7c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f0730a41640 1 -- 172.21.15.110:0/271118072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 msgr2=0x55befae837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f0730a41640 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f0730a41640 1 -- 172.21.15.110:0/271118072 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55befabb6620 con 0x55befae71e50 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f0730a41640 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae7c270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0724010010 tx=0x7f07240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efbc664f34d3c7e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f070bfff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f070bfff640 1 -- 172.21.15.110:0/271118072 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724016070 con 0x55befae71e50 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f071c086070 con 0x55befae71e50 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.047+0000 7f070bfff640 1 -- 172.21.15.110:0/271118072 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f072401bd00 con 0x55befae71e50 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f070bfff640 1 -- 172.21.15.110:0/271118072 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f072402e070 con 0x55befae71e50 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f07359ba740 10 client.24916 fetch_fsmap learned FSMap version 25 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55befae97c90 con 0x55befae71e50 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f070bfff640 1 -- 172.21.15.110:0/271118072 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f071c086070 con 0x55befae71e50 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f070bfff640 1 -- 172.21.15.110:0/271118072 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 25) ==== 8350+0+0 (secure 0 0 0) 0x7f072401c560 con 0x55befae71e50 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f07359ba740 10 client.24916 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:03:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f07359ba740 10 client.24916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:03:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f07359ba740 20 client.24916 populate_metadata read hostname 'smithi110' 2024-04-25T00:03:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f07359ba740 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55befae99390 0x55befae9b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f07359ba740 4 client.24916 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.048+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55befade31f0 con 0x55befae99390 2024-04-25T00:03:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.049+0000 7f072bfff640 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55befae99390 0x55befae9b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.049+0000 7f070bfff640 10 client.24916 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.049+0000 7f072bfff640 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55befae99390 0x55befae9b850 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd044830d430c03d server_cookie=f37dcd1cc256ed48 in_seq=0 out_seq=0 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f070bfff640 1 -- 172.21.15.110:0/271118072 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55befade31f0 con 0x55befae99390 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f070bfff640 10 client.24916 handle_command_reply: tid=0 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 client.24916 shutdown 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 msgr2=0x55befae7c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae7c270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0724010010 tx=0x7f07240046c0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 shutdown_connections 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55befae99390 0x55befae9b850 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f070bfff640 0 client.24916 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae7c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 --2- 172.21.15.110:0/271118072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae71e50 0x55befae7c270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 >> 172.21.15.110:0/271118072 conn(0x55befae6d4c0 msgr2=0x55befae6c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.050+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 shutdown_connections 2024-04-25T00:03:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.051+0000 7f07359ba740 1 -- 172.21.15.110:0/271118072 wait complete. 2024-04-25T00:03:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.051+0000 7f07359ba740 20 client.24916 trim_cache size 0 max 16384 2024-04-25T00:03:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.077+0000 7f07359ba740 1 Processor -- start 2024-04-25T00:03:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.077+0000 7f07359ba740 1 -- start start 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.077+0000 7f07359ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae71e50 0x55befae7be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.077+0000 7f07359ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae72840 0x55befae837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f07359ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae7c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f0730a41640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae71e50 0x55befae7be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f0730a41640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae71e50 0x55befae7be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47696/0 (socket says 172.21.15.110:47696) 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f0730a41640 1 -- 172.21.15.110:0/781219603 learned_addr learned my addr 172.21.15.110:0/781219603 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f07359ba740 1 -- 172.21.15.110:0/781219603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f072c1a9640 con 0x55befae779b0 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f07359ba740 1 -- 172.21.15.110:0/781219603 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f072c1a6ee0 con 0x55befae71e50 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f07359ba740 1 -- 172.21.15.110:0/781219603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f072c071c00 con 0x55befae72840 2024-04-25T00:03:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f072bfff640 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae72840 0x55befae837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f0730a41640 1 -- 172.21.15.110:0/781219603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae72840 msgr2=0x55befae837b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f0730a41640 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae72840 0x55befae837b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f0730a41640 1 -- 172.21.15.110:0/781219603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 msgr2=0x55befae7c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.078+0000 7f0731242640 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae7c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.079+0000 7f0730a41640 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae7c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.079+0000 7f0730a41640 1 -- 172.21.15.110:0/781219603 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55befae6d060 con 0x55befae71e50 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.079+0000 7f0731242640 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae7c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.079+0000 7f0730a41640 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae71e50 0x55befae7be80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0724010ac0 tx=0x7f072401cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ccf3f8501a180d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.079+0000 7f07227fc640 1 -- 172.21.15.110:0/781219603 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724040070 con 0x55befae71e50 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.079+0000 7f07227fc640 1 -- 172.21.15.110:0/781219603 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f072401c420 con 0x55befae71e50 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.079+0000 7f07227fc640 1 -- 172.21.15.110:0/781219603 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724016040 con 0x55befae71e50 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.080+0000 7f07359ba740 1 -- 172.21.15.110:0/781219603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae71e50 msgr2=0x55befae7be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.080+0000 7f07359ba740 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae71e50 0x55befae7be80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0724010ac0 tx=0x7f072401cdd0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.081+0000 7f07359ba740 1 -- 172.21.15.110:0/781219603 shutdown_connections 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.081+0000 7f07359ba740 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae779b0 0x55befae7c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.081+0000 7f07359ba740 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae72840 0x55befae837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.081+0000 7f07359ba740 1 --2- 172.21.15.110:0/781219603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae71e50 0x55befae7be80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.081+0000 7f07359ba740 1 -- 172.21.15.110:0/781219603 >> 172.21.15.110:0/781219603 conn(0x55befae6d4c0 msgr2=0x55befae6d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.081+0000 7f07359ba740 1 -- 172.21.15.110:0/781219603 shutdown_connections 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.081+0000 7f07359ba740 1 -- 172.21.15.110:0/781219603 wait complete. 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.081+0000 7f07359ba740 1 -- start start 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.082+0000 7f07359ba740 1 Processor -- start 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.082+0000 7f07359ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae71e50 0x55befae80840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.082+0000 7f0730a41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae71e50 0x55befae80840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.082+0000 7f07359ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae80d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.082+0000 7f07359ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae779b0 0x55befae81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.082+0000 7f07359ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f072c072270 con 0x55befae71e50 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f0730a41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae71e50 0x55befae80840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35884/0 (socket says 172.21.15.110:35884) 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f0730a41640 1 -- 172.21.15.110:0/2687396337 learned_addr learned my addr 172.21.15.110:0/2687396337 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f072c0736e0 con 0x55befae72840 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f072c072940 con 0x55befae779b0 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f072bfff640 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae80d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f0731242640 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae779b0 0x55befae81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f07359ba740 10 client.0 _ll_register_callbacks cb 0x7f072400a8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f072bfff640 1 -- 172.21.15.110:0/2687396337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae779b0 msgr2=0x55befae81d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f072bfff640 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae779b0 0x55befae81d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f072bfff640 1 -- 172.21.15.110:0/2687396337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae71e50 msgr2=0x55befae80840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f072bfff640 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae71e50 0x55befae80840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f072bfff640 1 -- 172.21.15.110:0/2687396337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55befae82310 con 0x55befae72840 2024-04-25T00:03:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f0730a41640 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae71e50 0x55befae80840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.083+0000 7f072bfff640 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae80d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07140093a0 tx=0x7f0714004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a159d4c19bf009e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.084+0000 7f070bfff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:03:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.084+0000 7f070bfff640 1 -- 172.21.15.110:0/2687396337 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714039040 con 0x55befae72840 2024-04-25T00:03:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.084+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f072401c420 con 0x55befae72840 2024-04-25T00:03:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.084+0000 7f070bfff640 1 -- 172.21.15.110:0/2687396337 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f07140043b0 con 0x55befae72840 2024-04-25T00:03:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.084+0000 7f070bfff640 1 -- 172.21.15.110:0/2687396337 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07140046c0 con 0x55befae72840 2024-04-25T00:03:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.084+0000 7f07359ba740 10 client.24944 fetch_fsmap learned FSMap version 25 2024-04-25T00:03:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.084+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55befae83610 con 0x55befae72840 2024-04-25T00:03:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f070bfff640 1 -- 172.21.15.110:0/2687396337 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f072401c420 con 0x55befae72840 2024-04-25T00:03:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f070bfff640 1 -- 172.21.15.110:0/2687396337 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8350+0+0 (secure 0 0 0) 0x7f0714003bd0 con 0x55befae72840 2024-04-25T00:03:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f07359ba740 10 client.24944 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:03:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f07359ba740 10 client.24944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:03:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f07359ba740 20 client.24944 populate_metadata read hostname 'smithi110' 2024-04-25T00:03:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f07359ba740 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55befae99390 0x55befaf01ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f07359ba740 4 client.24944 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:03:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55befaddd540 con 0x55befae99390 2024-04-25T00:03:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.085+0000 7f0730a41640 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55befae99390 0x55befaf01ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.086+0000 7f0730a41640 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55befae99390 0x55befaf01ce0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17fd9c54b64f09d5 server_cookie=82d9bd39bec770e2 in_seq=0 out_seq=0 2024-04-25T00:03:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.086+0000 7f070bfff640 10 client.24944 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.100+0000 7f070bfff640 1 -- 172.21.15.110:0/2687396337 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55befaddd540 con 0x55befae99390 2024-04-25T00:03:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.100+0000 7f070bfff640 10 client.24944 handle_command_reply: tid=0 2024-04-25T00:03:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.100+0000 7f07359ba740 1 client.24944 shutdown 2024-04-25T00:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 msgr2=0x55befae80d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae80d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07140093a0 tx=0x7f0714004040 comp rx=0 tx=0).stop 2024-04-25T00:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 shutdown_connections 2024-04-25T00:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55befae99390 0x55befaf01ce0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f070bfff640 0 client.24944 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55befae779b0 0x55befae81d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55befae72840 0x55befae80d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 --2- 172.21.15.110:0/2687396337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55befae71e50 0x55befae80840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 >> 172.21.15.110:0/2687396337 conn(0x55befae6d4c0 msgr2=0x55befae6bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 shutdown_connections 2024-04-25T00:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 1 -- 172.21.15.110:0/2687396337 wait complete. 2024-04-25T00:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.101+0000 7f07359ba740 20 client.24944 trim_cache size 0 max 16384 2024-04-25T00:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.102+0000 7f0732ca5640 1 -- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f07180a2cb0 msgr2=0x7f07180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.102+0000 7f0732ca5640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f07180a2cb0 0x7f07180a5170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f072c1016e0 tx=0x7f0724005f90 comp rx=0 tx=0).stop 2024-04-25T00:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.102+0000 7f0732ca5640 1 -- 172.21.15.110:0/2329996350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c1a3620 msgr2=0x7f072c1a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.102+0000 7f0732ca5640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c1a3620 0x7f072c1a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f071c00efc0 tx=0x7f071c00c800 comp rx=0 tx=0).stop 2024-04-25T00:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.103+0000 7f0732ca5640 1 -- 172.21.15.110:0/2329996350 shutdown_connections 2024-04-25T00:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.103+0000 7f0732ca5640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f07180a2cb0 0x7f07180a5170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.103+0000 7f0732ca5640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c1a3620 0x7f072c1a5a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.103+0000 7f0732ca5640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f072c102240 0x7f072c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.103+0000 7f0732ca5640 1 --2- 172.21.15.110:0/2329996350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f072c100690 0x7f072c19e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.103+0000 7f0732ca5640 1 -- 172.21.15.110:0/2329996350 >> 172.21.15.110:0/2329996350 conn(0x7f072c0fbc00 msgr2=0x7f072c104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.103+0000 7f0732ca5640 1 -- 172.21.15.110:0/2329996350 shutdown_connections 2024-04-25T00:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:13.103+0000 7f0732ca5640 1 -- 172.21.15.110:0/2329996350 wait complete. 2024-04-25T00:03:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:13 smithi188 ceph-mon[33285]: pgmap v386: 97 pgs: 97 active+clean; 954 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 42 MiB/s wr, 140 op/s 2024-04-25T00:03:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:13 smithi110 ceph-mon[29187]: pgmap v386: 97 pgs: 97 active+clean; 954 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 42 MiB/s wr, 140 op/s 2024-04-25T00:03:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:13 smithi173 ceph-mon[32402]: pgmap v386: 97 pgs: 97 active+clean; 954 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 42 MiB/s wr, 140 op/s 2024-04-25T00:03:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:15 smithi188 ceph-mon[33285]: pgmap v387: 97 pgs: 97 active+clean; 927 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 35 MiB/s wr, 269 op/s 2024-04-25T00:03:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:15 smithi110 ceph-mon[29187]: pgmap v387: 97 pgs: 97 active+clean; 927 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 35 MiB/s wr, 269 op/s 2024-04-25T00:03:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:15 smithi173 ceph-mon[32402]: pgmap v387: 97 pgs: 97 active+clean; 927 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 35 MiB/s wr, 269 op/s 2024-04-25T00:03:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 10.0 s, 1550.4 tps, lat 64.472 ms stddev 27.914 2024-04-25T00:03:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:17 smithi188 ceph-mon[33285]: pgmap v388: 97 pgs: 97 active+clean; 927 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 23 MiB/s wr, 311 op/s 2024-04-25T00:03:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:17 smithi110 ceph-mon[29187]: pgmap v388: 97 pgs: 97 active+clean; 927 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 23 MiB/s wr, 311 op/s 2024-04-25T00:03:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:17 smithi173 ceph-mon[32402]: pgmap v388: 97 pgs: 97 active+clean; 927 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 23 MiB/s wr, 311 op/s 2024-04-25T00:03:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:19 smithi188 ceph-mon[33285]: pgmap v389: 97 pgs: 97 active+clean; 930 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 25 MiB/s wr, 840 op/s 2024-04-25T00:03:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:19 smithi110 ceph-mon[29187]: pgmap v389: 97 pgs: 97 active+clean; 930 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 25 MiB/s wr, 840 op/s 2024-04-25T00:03:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:19 smithi173 ceph-mon[32402]: pgmap v389: 97 pgs: 97 active+clean; 930 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 25 MiB/s wr, 840 op/s 2024-04-25T00:03:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 15.0 s, 1593.4 tps, lat 62.783 ms stddev 17.443 2024-04-25T00:03:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:21 smithi188 ceph-mon[33285]: pgmap v390: 97 pgs: 97 active+clean; 930 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-04-25T00:03:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:21 smithi110 ceph-mon[29187]: pgmap v390: 97 pgs: 97 active+clean; 930 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-04-25T00:03:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:21 smithi173 ceph-mon[32402]: pgmap v390: 97 pgs: 97 active+clean; 930 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-04-25T00:03:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:23 smithi188 ceph-mon[33285]: pgmap v391: 97 pgs: 97 active+clean; 930 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 1.39k op/s 2024-04-25T00:03:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:23 smithi110 ceph-mon[29187]: pgmap v391: 97 pgs: 97 active+clean; 930 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 1.39k op/s 2024-04-25T00:03:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:23 smithi173 ceph-mon[32402]: pgmap v391: 97 pgs: 97 active+clean; 930 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 1.39k op/s 2024-04-25T00:03:26.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:25 smithi188 ceph-mon[33285]: pgmap v392: 97 pgs: 97 active+clean; 931 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-04-25T00:03:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:25 smithi110 ceph-mon[29187]: pgmap v392: 97 pgs: 97 active+clean; 931 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-04-25T00:03:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:25 smithi173 ceph-mon[32402]: pgmap v392: 97 pgs: 97 active+clean; 931 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-04-25T00:03:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.229+0000 7f6277f63640 1 -- 172.21.15.110:0/3187599952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 msgr2=0x7f62700ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.229+0000 7f6277f63640 1 --2- 172.21.15.110:0/3187599952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 0x7f62700ffeb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6264009930 tx=0x7f626402a420 comp rx=0 tx=0).stop 2024-04-25T00:03:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.229+0000 7f6277f63640 1 -- 172.21.15.110:0/3187599952 shutdown_connections 2024-04-25T00:03:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.229+0000 7f6277f63640 1 --2- 172.21.15.110:0/3187599952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6270103420 0x7f6270109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.230+0000 7f6277f63640 1 --2- 172.21.15.110:0/3187599952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6270102a60 0x7f6270102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.230+0000 7f6277f63640 1 --2- 172.21.15.110:0/3187599952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 0x7f62700ffeb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.230+0000 7f6277f63640 1 -- 172.21.15.110:0/3187599952 >> 172.21.15.110:0/3187599952 conn(0x7f62700fb3c0 msgr2=0x7f62700fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.230+0000 7f6277f63640 1 -- 172.21.15.110:0/3187599952 shutdown_connections 2024-04-25T00:03:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.230+0000 7f6277f63640 1 -- 172.21.15.110:0/3187599952 wait complete. 2024-04-25T00:03:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6277f63640 1 Processor -- start 2024-04-25T00:03:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6277f63640 1 -- start start 2024-04-25T00:03:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6277f63640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 0x7f6270199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6277f63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6270102a60 0x7f627019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6277f63640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6270103420 0x7f62701a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6277f63640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f627010d360 con 0x7f62700ffa90 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6277f63640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f627010d1e0 con 0x7f6270103420 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6277f63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f627010d4e0 con 0x7f6270102a60 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f6275cff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 0x7f6270199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f62754fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6270102a60 0x7f627019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f62754fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6270102a60 0x7f627019a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:40680/0 (socket says 172.21.15.110:40680) 2024-04-25T00:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.231+0000 7f62754fe640 1 -- 172.21.15.110:0/3061780437 learned_addr learned my addr 172.21.15.110:0/3061780437 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.232+0000 7f6276500640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6270103420 0x7f62701a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.232+0000 7f62754fe640 1 -- 172.21.15.110:0/3061780437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6270103420 msgr2=0x7f62701a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.232+0000 7f62754fe640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6270103420 0x7f62701a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.232+0000 7f62754fe640 1 -- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 msgr2=0x7f6270199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.232+0000 7f62754fe640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 0x7f6270199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.232+0000 7f62754fe640 1 -- 172.21.15.110:0/3061780437 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62701a17b0 con 0x7f6270102a60 2024-04-25T00:03:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.232+0000 7f6275cff640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 0x7f6270199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.232+0000 7f62754fe640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6270102a60 0x7f627019a0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f626c00e9f0 tx=0x7f626c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.233+0000 7f6262ffd640 1 -- 172.21.15.110:0/3061780437 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c00cd80 con 0x7f6270102a60 2024-04-25T00:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.233+0000 7f6262ffd640 1 -- 172.21.15.110:0/3061780437 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f626c0054d0 con 0x7f6270102a60 2024-04-25T00:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.233+0000 7f6262ffd640 1 -- 172.21.15.110:0/3061780437 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c010490 con 0x7f6270102a60 2024-04-25T00:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.233+0000 7f6277f63640 1 -- 172.21.15.110:0/3061780437 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62701a1aa0 con 0x7f6270102a60 2024-04-25T00:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.233+0000 7f6277f63640 1 -- 172.21.15.110:0/3061780437 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62701a1f40 con 0x7f6270102a60 2024-04-25T00:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.236+0000 7f627ac75740 1 Processor -- start 2024-04-25T00:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.236+0000 7f627ac75740 1 -- start start 2024-04-25T00:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.236+0000 7f6262ffd640 1 -- 172.21.15.110:0/3061780437 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f626c005020 con 0x7f6270102a60 2024-04-25T00:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f627ac75740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213cf660 0x5568213cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f627ac75740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d0050 0x5568213d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f627ac75740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d51c0 0x5568213d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f627ac75740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55682132acb0 con 0x5568213cf660 2024-04-25T00:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f627ac75740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5568212d35b0 con 0x5568213d0050 2024-04-25T00:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f627ac75740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556821326c70 con 0x5568213d51c0 2024-04-25T00:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f62754fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d51c0 0x5568213d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f62754fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d51c0 0x5568213d7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:40684/0 (socket says 172.21.15.110:40684) 2024-04-25T00:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f62754fe640 1 -- 172.21.15.110:0/2916012940 learned_addr learned my addr 172.21.15.110:0/2916012940 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f6275cff640 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d0050 0x5568213d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f6276500640 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213cf660 0x5568213cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f6262ffd640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f625c0a2cb0 0x7f625c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f6262ffd640 1 -- 172.21.15.110:0/3061780437 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f626c0cf430 con 0x7f6270102a60 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f62754fe640 1 -- 172.21.15.110:0/2916012940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d0050 msgr2=0x5568213d4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f62754fe640 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d0050 0x5568213d4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.237+0000 7f62754fe640 1 -- 172.21.15.110:0/2916012940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213cf660 msgr2=0x5568213cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.238+0000 7f62754fe640 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213cf660 0x5568213cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.238+0000 7f62754fe640 1 -- 172.21.15.110:0/2916012940 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682103f210 con 0x5568213d51c0 2024-04-25T00:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.238+0000 7f6276500640 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213cf660 0x5568213cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.238+0000 7f62754fe640 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d51c0 0x5568213d7650 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f626c0ccf00 tx=0x7f626c098030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f800334fcabc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.238+0000 7f6276500640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f625c0a2cb0 0x7f625c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.238+0000 7f625bfff640 1 -- 172.21.15.110:0/2916012940 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c09b5c0 con 0x5568213d51c0 2024-04-25T00:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.238+0000 7f625bfff640 1 -- 172.21.15.110:0/2916012940 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f626c0b1020 con 0x5568213d51c0 2024-04-25T00:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.238+0000 7f625bfff640 1 -- 172.21.15.110:0/2916012940 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c0a0010 con 0x5568213d51c0 2024-04-25T00:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 -- 172.21.15.110:0/2916012940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d51c0 msgr2=0x5568213d7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d51c0 0x5568213d7650 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f626c0ccf00 tx=0x7f626c098030 comp rx=0 tx=0).stop 2024-04-25T00:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f6276500640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f625c0a2cb0 0x7f625c0a5170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x5568213d08b0 tx=0x7f6268004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 -- 172.21.15.110:0/2916012940 shutdown_connections 2024-04-25T00:03:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d51c0 0x5568213d7650 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d0050 0x5568213d4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 --2- 172.21.15.110:0/2916012940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213cf660 0x5568213cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 -- 172.21.15.110:0/2916012940 >> 172.21.15.110:0/2916012940 conn(0x5568213cacd0 msgr2=0x5568213cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 -- 172.21.15.110:0/2916012940 shutdown_connections 2024-04-25T00:03:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.239+0000 7f627ac75740 1 -- 172.21.15.110:0/2916012940 wait complete. 2024-04-25T00:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 1 -- start start 2024-04-25T00:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 1 Processor -- start 2024-04-25T00:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213d9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5568213d80b0 con 0x5568213d51c0 2024-04-25T00:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5568212e6b50 con 0x5568213cf660 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5568213d8230 con 0x5568213d0050 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f627ac75740 10 client.0 _ll_register_callbacks cb 0x556821321ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f6275cff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f6275cff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213d9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:52706/0 (socket says 172.21.15.110:52706) 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f6275cff640 1 -- 172.21.15.110:0/2166260209 learned_addr learned my addr 172.21.15.110:0/2166260209 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f6276500640 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.240+0000 7f62754fe640 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213d9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f6275cff640 1 -- 172.21.15.110:0/2166260209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 msgr2=0x5568213d9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f6275cff640 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213d9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f6275cff640 1 -- 172.21.15.110:0/2166260209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 msgr2=0x5568213e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f6275cff640 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f6275cff640 1 -- 172.21.15.110:0/2166260209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682110b620 con 0x5568213cf660 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f6276500640 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f6275cff640 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213d9a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6270069d10 tx=0x7f6264005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2277578f884e64a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f62597fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f62597fa640 1 -- 172.21.15.110:0/2166260209 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264035040 con 0x5568213cf660 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f627ac75740 1 -- 172.21.15.110:0/2166260209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f626c0b1020 con 0x5568213cf660 2024-04-25T00:03:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f62597fa640 1 -- 172.21.15.110:0/2166260209 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f62640048c0 con 0x5568213cf660 2024-04-25T00:03:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f62597fa640 1 -- 172.21.15.110:0/2166260209 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f626404c070 con 0x5568213cf660 2024-04-25T00:03:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f627ac75740 10 client.24956 fetch_fsmap learned FSMap version 25 2024-04-25T00:03:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.241+0000 7f627ac75740 1 -- 172.21.15.110:0/2166260209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5568213e43c0 con 0x5568213cf660 2024-04-25T00:03:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f62597fa640 1 -- 172.21.15.110:0/2166260209 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f626c0b1020 con 0x5568213cf660 2024-04-25T00:03:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f62597fa640 1 -- 172.21.15.110:0/2166260209 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8350+0+0 (secure 0 0 0) 0x7f6264003c30 con 0x5568213cf660 2024-04-25T00:03:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f627ac75740 10 client.24956 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:03:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f627ac75740 10 client.24956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:03:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f627ac75740 20 client.24956 populate_metadata read hostname 'smithi110' 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f627ac75740 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5568213e5e70 0x5568213e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f627ac75740 4 client.24956 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f627ac75740 1 -- 172.21.15.110:0/2166260209 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568213381f0 con 0x5568213e5e70 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.242+0000 7f62754fe640 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5568213e5e70 0x5568213e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.243+0000 7f62754fe640 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5568213e5e70 0x5568213e8310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26db994b1aec5bd6 server_cookie=7f30769e90b771ac in_seq=0 out_seq=0 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.243+0000 7f62597fa640 10 client.24956 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.243+0000 7f62597fa640 1 -- 172.21.15.110:0/2166260209 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568213381f0 con 0x5568213e5e70 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.243+0000 7f62597fa640 10 client.24956 handle_command_reply: tid=0 2024-04-25T00:03:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 client.24956 shutdown 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 -- 172.21.15.110:0/2166260209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 msgr2=0x5568213d9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213d9a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6270069d10 tx=0x7f6264005270 comp rx=0 tx=0).stop 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 -- 172.21.15.110:0/2166260209 shutdown_connections 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5568213e5e70 0x5568213e8310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213d9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 --2- 172.21.15.110:0/2166260209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213d9a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 -- 172.21.15.110:0/2166260209 >> 172.21.15.110:0/2166260209 conn(0x5568213cacd0 msgr2=0x5568213c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f62597fa640 0 client.24956 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 -- 172.21.15.110:0/2166260209 shutdown_connections 2024-04-25T00:03:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 1 -- 172.21.15.110:0/2166260209 wait complete. 2024-04-25T00:03:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.244+0000 7f627ac75740 20 client.24956 trim_cache size 0 max 16384 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.269+0000 7f627ac75740 1 Processor -- start 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.269+0000 7f627ac75740 1 -- start start 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.269+0000 7f627ac75740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213cf660 0x5568213cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.269+0000 7f627ac75740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d0050 0x5568213e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.269+0000 7f627ac75740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d51c0 0x5568213d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.269+0000 7f627ac75740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6270072150 con 0x5568213d0050 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.269+0000 7f627ac75740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f627006f9f0 con 0x5568213d51c0 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.269+0000 7f627ac75740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f627006d860 con 0x5568213cf660 2024-04-25T00:03:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f62754fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d0050 0x5568213e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f62754fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d0050 0x5568213e0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43162/0 (socket says 172.21.15.110:43162) 2024-04-25T00:03:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f62754fe640 1 -- 172.21.15.110:0/4107940917 learned_addr learned my addr 172.21.15.110:0/4107940917 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f6276500640 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d51c0 0x5568213d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f6275cff640 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213cf660 0x5568213cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f6276500640 1 -- 172.21.15.110:0/4107940917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213cf660 msgr2=0x5568213cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f6276500640 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213cf660 0x5568213cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f6276500640 1 -- 172.21.15.110:0/4107940917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d0050 msgr2=0x5568213e0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f6276500640 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d0050 0x5568213e0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f6276500640 1 -- 172.21.15.110:0/4107940917 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568213d47a0 con 0x5568213d51c0 2024-04-25T00:03:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f62754fe640 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d0050 0x5568213e0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.270+0000 7f6276500640 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d51c0 0x5568213d9030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6268009fd0 tx=0x7f6268027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411704fa9834e946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.271+0000 7f625b7fe640 1 -- 172.21.15.110:0/4107940917 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268010660 con 0x5568213d51c0 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.271+0000 7f625b7fe640 1 -- 172.21.15.110:0/4107940917 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6268010cb0 con 0x5568213d51c0 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.271+0000 7f625b7fe640 1 -- 172.21.15.110:0/4107940917 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268011d60 con 0x5568213d51c0 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.271+0000 7f6275cff640 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213cf660 0x5568213cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 -- 172.21.15.110:0/4107940917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d51c0 msgr2=0x5568213d9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d51c0 0x5568213d9030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6268009fd0 tx=0x7f6268027000 comp rx=0 tx=0).stop 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 -- 172.21.15.110:0/4107940917 shutdown_connections 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213d51c0 0x5568213d9030 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d0050 0x5568213e0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 --2- 172.21.15.110:0/4107940917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213cf660 0x5568213cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 -- 172.21.15.110:0/4107940917 >> 172.21.15.110:0/4107940917 conn(0x5568213cacd0 msgr2=0x5568213c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 -- 172.21.15.110:0/4107940917 shutdown_connections 2024-04-25T00:03:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 -- 172.21.15.110:0/4107940917 wait complete. 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.272+0000 7f627ac75740 1 -- start start 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f627ac75740 1 Processor -- start 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f627ac75740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213de020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f627ac75740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213de560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f627ac75740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213df500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f627ac75740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f627006deb0 con 0x5568213d51c0 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f627ac75740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f627006f320 con 0x5568213cf660 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f627ac75740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f627006e580 con 0x5568213d0050 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f627ac75740 10 client.0 _ll_register_callbacks cb 0x7f6264009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f62754fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213de560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f6276500640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f62754fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213de560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:40710/0 (socket says 172.21.15.110:40710) 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f62754fe640 1 -- 172.21.15.110:0/3026440740 learned_addr learned my addr 172.21.15.110:0/3026440740 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f6275cff640 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f6276500640 1 -- 172.21.15.110:0/3026440740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 msgr2=0x5568213de560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f6276500640 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213de560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.273+0000 7f6276500640 1 -- 172.21.15.110:0/3026440740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 msgr2=0x5568213de020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.274+0000 7f6276500640 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213de020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.274+0000 7f6276500640 1 -- 172.21.15.110:0/3026440740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568213dfaf0 con 0x5568213d51c0 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.274+0000 7f62597fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.274+0000 7f6276500640 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213df500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6268009fd0 tx=0x7f6268009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ece15c77cf393f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.274+0000 7f62597fa640 1 -- 172.21.15.110:0/3026440740 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268044070 con 0x5568213d51c0 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.274+0000 7f62597fa640 1 -- 172.21.15.110:0/3026440740 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6268010460 con 0x5568213d51c0 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.274+0000 7f627ac75740 1 -- 172.21.15.110:0/3026440740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6268010cb0 con 0x5568213d51c0 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f62597fa640 1 -- 172.21.15.110:0/3026440740 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6268050050 con 0x5568213d51c0 2024-04-25T00:03:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f627ac75740 10 client.15384 fetch_fsmap learned FSMap version 25 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f627ac75740 1 -- 172.21.15.110:0/3026440740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5568213e0d50 con 0x5568213d51c0 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f62597fa640 1 -- 172.21.15.110:0/3026440740 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6268010cb0 con 0x5568213d51c0 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f62597fa640 1 -- 172.21.15.110:0/3026440740 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8350+0+0 (secure 0 0 0) 0x7f626803b960 con 0x5568213d51c0 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f627ac75740 10 client.15384 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f627ac75740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f627ac75740 20 client.15384 populate_metadata read hostname 'smithi110' 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f627ac75740 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55682144c710 0x55682144eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.275+0000 7f627ac75740 4 client.15384 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.276+0000 7f627ac75740 1 -- 172.21.15.110:0/3026440740 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556821332540 con 0x55682144c710 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.276+0000 7f6275cff640 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55682144c710 0x55682144eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.276+0000 7f6275cff640 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55682144c710 0x55682144eb30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8cb5e89fc64626 server_cookie=1d566ce8b4b71619 in_seq=0 out_seq=0 2024-04-25T00:03:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.276+0000 7f62597fa640 10 client.15384 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.292+0000 7f62597fa640 1 -- 172.21.15.110:0/3026440740 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556821332540 con 0x55682144c710 2024-04-25T00:03:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.292+0000 7f62597fa640 10 client.15384 handle_command_reply: tid=0 2024-04-25T00:03:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.292+0000 7f627ac75740 1 client.15384 shutdown 2024-04-25T00:03:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.292+0000 7f627ac75740 1 -- 172.21.15.110:0/3026440740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 msgr2=0x5568213df500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.292+0000 7f627ac75740 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213df500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6268009fd0 tx=0x7f6268009750 comp rx=0 tx=0).stop 2024-04-25T00:03:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 1 -- 172.21.15.110:0/3026440740 shutdown_connections 2024-04-25T00:03:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55682144c710 0x55682144eb30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568213d51c0 0x5568213df500 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f62597fa640 0 client.15384 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5568213d0050 0x5568213de560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 1 --2- 172.21.15.110:0/3026440740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568213cf660 0x5568213de020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 1 -- 172.21.15.110:0/3026440740 >> 172.21.15.110:0/3026440740 conn(0x5568213cacd0 msgr2=0x5568213c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 1 -- 172.21.15.110:0/3026440740 shutdown_connections 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 1 -- 172.21.15.110:0/3026440740 wait complete. 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.293+0000 7f627ac75740 20 client.15384 trim_cache size 0 max 16384 2024-04-25T00:03:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:03:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:03:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:03:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:03:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 -- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f625c0a2cb0 msgr2=0x7f625c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f625c0a2cb0 0x7f625c0a5170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x5568213d08b0 tx=0x7f6268004a40 comp rx=0 tx=0).stop 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 -- 172.21.15.110:0/3061780437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6270102a60 msgr2=0x7f627019a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6270102a60 0x7f627019a0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f626c00e9f0 tx=0x7f626c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 -- 172.21.15.110:0/3061780437 shutdown_connections 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f625c0a2cb0 0x7f625c0a5170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6270103420 0x7f62701a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6270102a60 0x7f627019a0d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 --2- 172.21.15.110:0/3061780437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f62700ffa90 0x7f6270199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 -- 172.21.15.110:0/3061780437 >> 172.21.15.110:0/3061780437 conn(0x7f62700fb3c0 msgr2=0x7f6270100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 -- 172.21.15.110:0/3061780437 shutdown_connections 2024-04-25T00:03:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:26.294+0000 7f6277f63640 1 -- 172.21.15.110:0/3061780437 wait complete. 2024-04-25T00:03:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 20.0 s, 1514.8 tps, lat 66.014 ms stddev 58.011 2024-04-25T00:03:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:27 smithi188 ceph-mon[33285]: pgmap v393: 97 pgs: 97 active+clean; 931 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-04-25T00:03:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:27 smithi110 ceph-mon[29187]: pgmap v393: 97 pgs: 97 active+clean; 931 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-04-25T00:03:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:27 smithi173 ceph-mon[32402]: pgmap v393: 97 pgs: 97 active+clean; 931 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-04-25T00:03:29.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:28 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:03:29.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:28 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:28 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:28 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:28 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:28 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:28 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:03:29.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:28 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:28 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:03:29.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:28 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:28 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:28 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:28 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:28 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:29.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:28 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:29 smithi188 ceph-mon[33285]: pgmap v394: 97 pgs: 97 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 2.11k op/s 2024-04-25T00:03:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:03:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:03:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:29 smithi110 ceph-mon[29187]: pgmap v394: 97 pgs: 97 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 2.11k op/s 2024-04-25T00:03:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:03:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:03:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:29 smithi173 ceph-mon[32402]: pgmap v394: 97 pgs: 97 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 2.11k op/s 2024-04-25T00:03:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:03:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:03:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:03:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 25.0 s, 1542.0 tps, lat 64.757 ms stddev 18.826 2024-04-25T00:03:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:31 smithi188 ceph-mon[33285]: pgmap v395: 97 pgs: 97 active+clean; 943 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.85k op/s 2024-04-25T00:03:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:31 smithi110 ceph-mon[29187]: pgmap v395: 97 pgs: 97 active+clean; 943 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.85k op/s 2024-04-25T00:03:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:31 smithi173 ceph-mon[32402]: pgmap v395: 97 pgs: 97 active+clean; 943 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 17 MiB/s wr, 1.85k op/s 2024-04-25T00:03:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:32 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:33.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:32 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:33.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:32 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:33 smithi188 ceph-mon[33285]: pgmap v396: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.90k op/s 2024-04-25T00:03:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:33 smithi110 ceph-mon[29187]: pgmap v396: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.90k op/s 2024-04-25T00:03:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:33 smithi173 ceph-mon[32402]: pgmap v396: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.90k op/s 2024-04-25T00:03:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:35 smithi188 ceph-mon[33285]: pgmap v397: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.92k op/s 2024-04-25T00:03:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:35 smithi110 ceph-mon[29187]: pgmap v397: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.92k op/s 2024-04-25T00:03:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:35 smithi173 ceph-mon[32402]: pgmap v397: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.92k op/s 2024-04-25T00:03:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 30.0 s, 1557.2 tps, lat 56.243 ms stddev 19.902 2024-04-25T00:03:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:37 smithi188 ceph-mon[33285]: pgmap v398: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.69k op/s 2024-04-25T00:03:38.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:37 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:37 smithi110 ceph-mon[29187]: pgmap v398: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.69k op/s 2024-04-25T00:03:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:37 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:37 smithi173 ceph-mon[32402]: pgmap v398: 97 pgs: 97 active+clean; 947 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.69k op/s 2024-04-25T00:03:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:37 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:03:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.432+0000 7fc88da14640 1 -- 172.21.15.110:0/1676793230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc880018290 msgr2=0x7fc8800a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.432+0000 7fc88da14640 1 --2- 172.21.15.110:0/1676793230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc880018290 0x7fc8800a61c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009900 tx=0x7fc87c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.433+0000 7fc88da14640 1 -- 172.21.15.110:0/1676793230 shutdown_connections 2024-04-25T00:03:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.433+0000 7fc88da14640 1 --2- 172.21.15.110:0/1676793230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8800ab400 0x7fc8800ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.433+0000 7fc88da14640 1 --2- 172.21.15.110:0/1676793230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8800a6700 0x7fc8800a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.433+0000 7fc88da14640 1 --2- 172.21.15.110:0/1676793230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc880018290 0x7fc8800a61c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.433+0000 7fc88da14640 1 -- 172.21.15.110:0/1676793230 >> 172.21.15.110:0/1676793230 conn(0x7fc8800a0520 msgr2=0x7fc8800a2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.433+0000 7fc88da14640 1 -- 172.21.15.110:0/1676793230 shutdown_connections 2024-04-25T00:03:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.433+0000 7fc88da14640 1 -- 172.21.15.110:0/1676793230 wait complete. 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88da14640 1 Processor -- start 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88da14640 1 -- start start 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88da14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc880018290 0x7fc880143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88da14640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8800a6700 0x7fc880143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88da14640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8800ab400 0x7fc88014abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88da14640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800affd0 con 0x7fc880018290 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88da14640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800afe50 con 0x7fc8800a6700 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88da14640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800b0150 con 0x7fc8800ab400 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc887fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8800a6700 0x7fc880143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88d213640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8800ab400 0x7fc88014abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc887fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8800a6700 0x7fc880143b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:35048/0 (socket says 172.21.15.110:35048) 2024-04-25T00:03:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc887fff640 1 -- 172.21.15.110:0/3941200968 learned_addr learned my addr 172.21.15.110:0/3941200968 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.434+0000 7fc88ca12640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc880018290 0x7fc880143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc88d213640 1 -- 172.21.15.110:0/3941200968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8800a6700 msgr2=0x7fc880143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc88d213640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8800a6700 0x7fc880143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc88d213640 1 -- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc880018290 msgr2=0x7fc880143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc88d213640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc880018290 0x7fc880143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc88d213640 1 -- 172.21.15.110:0/3941200968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88014b220 con 0x7fc8800ab400 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc88ca12640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc880018290 0x7fc880143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc88d213640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8800ab400 0x7fc88014abb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc88804f7d0 tx=0x7fc888070850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc885ffb640 1 -- 172.21.15.110:0/3941200968 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88806d5b0 con 0x7fc8800ab400 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc885ffb640 1 -- 172.21.15.110:0/3941200968 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc88806dc00 con 0x7fc8800ab400 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.435+0000 7fc88da14640 1 -- 172.21.15.110:0/3941200968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88014b510 con 0x7fc8800ab400 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.436+0000 7fc885ffb640 1 -- 172.21.15.110:0/3941200968 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88807bcd0 con 0x7fc8800ab400 2024-04-25T00:03:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.436+0000 7fc88da14640 1 -- 172.21.15.110:0/3941200968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88014b9e0 con 0x7fc8800ab400 2024-04-25T00:03:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.439+0000 7fc891988740 1 Processor -- start 2024-04-25T00:03:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc885ffb640 1 -- 172.21.15.110:0/3941200968 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc88807b420 con 0x7fc8800ab400 2024-04-25T00:03:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc891988740 1 -- start start 2024-04-25T00:03:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc891988740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120db450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc891988740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc891988740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120e0b90 0x555b120e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc891988740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b1203ecb0 con 0x555b120db030 2024-04-25T00:03:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc891988740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555b11fe75b0 con 0x555b120e0b90 2024-04-25T00:03:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc891988740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555b1203ac70 con 0x555b120dba20 2024-04-25T00:03:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc88ca12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc88ca12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:42790/0 (socket says 172.21.15.110:42790) 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.441+0000 7fc88ca12640 1 -- 172.21.15.110:0/399596090 learned_addr learned my addr 172.21.15.110:0/399596090 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.442+0000 7fc88ca12640 1 -- 172.21.15.110:0/399596090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120e0b90 msgr2=0x555b120e3020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.442+0000 7fc88ca12640 1 --2- 172.21.15.110:0/399596090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120e0b90 0x555b120e3020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.442+0000 7fc88ca12640 1 -- 172.21.15.110:0/399596090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 msgr2=0x555b120db450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.442+0000 7fc88ca12640 1 --2- 172.21.15.110:0/399596090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120db450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.442+0000 7fc88ca12640 1 -- 172.21.15.110:0/399596090 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b11d53210 con 0x555b120dba20 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.442+0000 7fc88ca12640 1 --2- 172.21.15.110:0/399596090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e0520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009900 tx=0x7fc87c0042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c1faccc56a6ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.442+0000 7fc885ffb640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8780a2d60 0x7fc8780a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.442+0000 7fc872ffd640 1 -- 172.21.15.110:0/399596090 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c035030 con 0x555b120dba20 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.443+0000 7fc887fff640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8780a2d60 0x7fc8780a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.443+0000 7fc872ffd640 1 -- 172.21.15.110:0/399596090 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc87c0044d0 con 0x555b120dba20 2024-04-25T00:03:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.443+0000 7fc872ffd640 1 -- 172.21.15.110:0/399596090 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c003850 con 0x555b120dba20 2024-04-25T00:03:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.443+0000 7fc885ffb640 1 -- 172.21.15.110:0/3941200968 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc888132560 con 0x7fc8800ab400 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc887fff640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8780a2d60 0x7fc8780a5220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b6c0 tx=0x7fc87400b650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 -- 172.21.15.110:0/399596090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 msgr2=0x555b120e0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 --2- 172.21.15.110:0/399596090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e0520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009900 tx=0x7fc87c0042e0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 -- 172.21.15.110:0/399596090 shutdown_connections 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 --2- 172.21.15.110:0/399596090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120e0b90 0x555b120e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 --2- 172.21.15.110:0/399596090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e0520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 --2- 172.21.15.110:0/399596090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120db450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 -- 172.21.15.110:0/399596090 >> 172.21.15.110:0/399596090 conn(0x555b120d66a0 msgr2=0x555b120d6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 -- 172.21.15.110:0/399596090 shutdown_connections 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 -- 172.21.15.110:0/399596090 wait complete. 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 -- start start 2024-04-25T00:03:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.444+0000 7fc891988740 1 Processor -- start 2024-04-25T00:03:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.445+0000 7fc891988740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120e5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.445+0000 7fc891988740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.445+0000 7fc891988740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.445+0000 7fc88ca12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.445+0000 7fc88d213640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.445+0000 7fc88ca12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120e5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:35054/0 (socket says 172.21.15.110:35054) 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.445+0000 7fc88ca12640 1 -- 172.21.15.110:0/1916650908 learned_addr learned my addr 172.21.15.110:0/1916650908 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.445+0000 7fc887fff640 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b120e3ae0 con 0x555b120e0b90 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555b11ffab50 con 0x555b120db030 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555b120e3c60 con 0x555b120dba20 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc891988740 10 client.0 _ll_register_callbacks cb 0x555b12035ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc88ca12640 1 -- 172.21.15.110:0/1916650908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 msgr2=0x555b120e5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc88ca12640 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc88ca12640 1 -- 172.21.15.110:0/1916650908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 msgr2=0x555b120ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc88ca12640 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc88ca12640 1 -- 172.21.15.110:0/1916650908 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b11e1f620 con 0x555b120db030 2024-04-25T00:03:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc88d213640 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc87c0044d0 con 0x555b120db030 2024-04-25T00:03:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc88ca12640 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120e5450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0092a0 tx=0x7fc87c005190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a49a0f866c38ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.446+0000 7fc867fff640 10 client.24971 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:03:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc867fff640 1 -- 172.21.15.110:0/1916650908 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c051060 con 0x555b120db030 2024-04-25T00:03:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc867fff640 1 -- 172.21.15.110:0/1916650908 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc87c031580 con 0x555b120db030 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc867fff640 1 -- 172.21.15.110:0/1916650908 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc87c035040 con 0x555b120db030 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc891988740 10 client.24971 fetch_fsmap learned FSMap version 27 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({fsmap=27}) -- 0x555b120efbc0 con 0x555b120db030 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc867fff640 1 -- 172.21.15.110:0/1916650908 <== mon.2 v2:172.21.15.173:3300/0 4 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fc87c04b030 con 0x555b120db030 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc891988740 10 client.24971 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc891988740 10 client.24971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc891988740 20 client.24971 populate_metadata read hostname 'smithi110' 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc891988740 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555b120f0f10 0x555b120f3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc891988740 4 client.24971 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.447+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b1204c1f0 con 0x555b120f0f10 2024-04-25T00:03:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.448+0000 7fc887fff640 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555b120f0f10 0x555b120f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.448+0000 7fc887fff640 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555b120f0f10 0x555b120f3300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d089315c7d5754 server_cookie=b0849ec4e3977633 in_seq=0 out_seq=0 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.448+0000 7fc867fff640 10 client.24971 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc867fff640 1 -- 172.21.15.110:0/1916650908 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b1204c1f0 con 0x555b120f0f10 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc867fff640 10 client.24971 handle_command_reply: tid=0 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 client.24971 shutdown 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 msgr2=0x555b120e5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120e5450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0092a0 tx=0x7fc87c005190 comp rx=0 tx=0).stop 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 shutdown_connections 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555b120f0f10 0x555b120f3300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc867fff640 0 client.24971 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120e5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 --2- 172.21.15.110:0/1916650908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120e5450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.449+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 >> 172.21.15.110:0/1916650908 conn(0x555b120d66a0 msgr2=0x555b120d58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.450+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 shutdown_connections 2024-04-25T00:03:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.450+0000 7fc891988740 1 -- 172.21.15.110:0/1916650908 wait complete. 2024-04-25T00:03:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.450+0000 7fc891988740 20 client.24971 trim_cache size 0 max 16384 2024-04-25T00:03:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.476+0000 7fc891988740 1 Processor -- start 2024-04-25T00:03:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc891988740 1 -- start start 2024-04-25T00:03:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc891988740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc891988740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc891988740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc891988740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc880157030 con 0x555b120e0b90 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc891988740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc880154870 con 0x555b120db030 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc891988740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801526c0 con 0x555b120dba20 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc88ca12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc88ca12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120d5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:35064/0 (socket says 172.21.15.110:35064) 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.477+0000 7fc88ca12640 1 -- 172.21.15.110:0/3452996986 learned_addr learned my addr 172.21.15.110:0/3452996986 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc88d213640 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc887fff640 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc88ca12640 1 -- 172.21.15.110:0/3452996986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 msgr2=0x555b120d5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc88ca12640 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120d5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc88ca12640 1 -- 172.21.15.110:0/3452996986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 msgr2=0x555b120ec990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc88ca12640 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc88ca12640 1 -- 172.21.15.110:0/3452996986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b120d61e0 con 0x555b120db030 2024-04-25T00:03:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc88ca12640 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120d5720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc87c006010 tx=0x7fc87c044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af60631a0495b499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc8727fc640 1 -- 172.21.15.110:0/3452996986 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c031580 con 0x555b120db030 2024-04-25T00:03:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc8727fc640 1 -- 172.21.15.110:0/3452996986 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc87c0044d0 con 0x555b120db030 2024-04-25T00:03:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.478+0000 7fc8727fc640 1 -- 172.21.15.110:0/3452996986 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c03bb40 con 0x555b120db030 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.479+0000 7fc891988740 1 -- 172.21.15.110:0/3452996986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 msgr2=0x555b120d5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.479+0000 7fc891988740 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120d5720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc87c006010 tx=0x7fc87c044000 comp rx=0 tx=0).stop 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.480+0000 7fc891988740 1 -- 172.21.15.110:0/3452996986 shutdown_connections 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.480+0000 7fc891988740 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120e0b90 0x555b120ec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.480+0000 7fc891988740 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120dba20 0x555b120d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.480+0000 7fc891988740 1 --2- 172.21.15.110:0/3452996986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120db030 0x555b120d5720 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.480+0000 7fc891988740 1 -- 172.21.15.110:0/3452996986 >> 172.21.15.110:0/3452996986 conn(0x555b120d66a0 msgr2=0x555b120d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.480+0000 7fc891988740 1 -- 172.21.15.110:0/3452996986 shutdown_connections 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.480+0000 7fc891988740 1 -- 172.21.15.110:0/3452996986 wait complete. 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.481+0000 7fc891988740 1 -- start start 2024-04-25T00:03:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.481+0000 7fc891988740 1 Processor -- start 2024-04-25T00:03:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc891988740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120ec930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc891988740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120dba20 0x555b120e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc891988740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120e0b90 0x555b120e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc891988740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc880152d30 con 0x555b120db030 2024-04-25T00:03:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc891988740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801541a0 con 0x555b120dba20 2024-04-25T00:03:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc891988740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc880153400 con 0x555b120e0b90 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88ca12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120ec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc891988740 10 client.0 _ll_register_callbacks cb 0x7fc87c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88ca12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120ec930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56034/0 (socket says 172.21.15.110:56034) 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88d213640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120e0b90 0x555b120e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88ca12640 1 -- 172.21.15.110:0/3508554488 learned_addr learned my addr 172.21.15.110:0/3508554488 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc887fff640 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120dba20 0x555b120e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88ca12640 1 -- 172.21.15.110:0/3508554488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120e0b90 msgr2=0x555b120e99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88ca12640 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120e0b90 0x555b120e99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88ca12640 1 -- 172.21.15.110:0/3508554488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120dba20 msgr2=0x555b120e93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88ca12640 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120dba20 0x555b120e93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.482+0000 7fc88ca12640 1 -- 172.21.15.110:0/3508554488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b12155470 con 0x555b120db030 2024-04-25T00:03:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc88ca12640 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120ec930 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009900 tx=0x7fc87c031f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cf7564d50af94c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc867fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc867fff640 1 -- 172.21.15.110:0/3508554488 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c063020 con 0x555b120db030 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc891988740 1 -- 172.21.15.110:0/3508554488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc87c0044d0 con 0x555b120db030 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc867fff640 1 -- 172.21.15.110:0/3508554488 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc87c03b430 con 0x555b120db030 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc867fff640 1 -- 172.21.15.110:0/3508554488 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc87c051030 con 0x555b120db030 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc891988740 10 client.15399 fetch_fsmap learned FSMap version 27 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc891988740 1 -- 172.21.15.110:0/3508554488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b12156720 con 0x555b120db030 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.483+0000 7fc867fff640 1 -- 172.21.15.110:0/3508554488 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc87c0044d0 con 0x555b120db030 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc867fff640 1 -- 172.21.15.110:0/3508554488 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fc87c04b070 con 0x555b120db030 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc891988740 10 client.15399 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc891988740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc891988740 20 client.15399 populate_metadata read hostname 'smithi110' 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc891988740 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555b12156fe0 0x555b12159400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc891988740 4 client.15399 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:03:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc891988740 1 -- 172.21.15.110:0/3508554488 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b12046540 con 0x555b12156fe0 2024-04-25T00:03:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc887fff640 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555b12156fe0 0x555b12159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc867fff640 10 client.15399 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.484+0000 7fc887fff640 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555b12156fe0 0x555b12159400 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8156f23c6119098e server_cookie=a229676c0fe6dcbf in_seq=0 out_seq=0 2024-04-25T00:03:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.501+0000 7fc867fff640 1 -- 172.21.15.110:0/3508554488 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b12046540 con 0x555b12156fe0 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.501+0000 7fc867fff640 10 client.15399 handle_command_reply: tid=0 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.501+0000 7fc891988740 1 client.15399 shutdown 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 -- 172.21.15.110:0/3508554488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 msgr2=0x555b120ec930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120ec930 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009900 tx=0x7fc87c031f30 comp rx=0 tx=0).stop 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 -- 172.21.15.110:0/3508554488 shutdown_connections 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555b12156fe0 0x555b12159400 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b120e0b90 0x555b120e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555b120dba20 0x555b120e93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 --2- 172.21.15.110:0/3508554488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b120db030 0x555b120ec930 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc867fff640 0 client.15399 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 -- 172.21.15.110:0/3508554488 >> 172.21.15.110:0/3508554488 conn(0x555b120d66a0 msgr2=0x555b120d58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 -- 172.21.15.110:0/3508554488 shutdown_connections 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 1 -- 172.21.15.110:0/3508554488 wait complete. 2024-04-25T00:03:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.502+0000 7fc891988740 20 client.15399 trim_cache size 0 max 16384 2024-04-25T00:03:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:03:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:03:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:03:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:03:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.503+0000 7fc88da14640 1 -- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8780a2d60 msgr2=0x7fc8780a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.503+0000 7fc88da14640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8780a2d60 0x7fc8780a5220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b6c0 tx=0x7fc87400b650 comp rx=0 tx=0).stop 2024-04-25T00:03:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.503+0000 7fc88da14640 1 -- 172.21.15.110:0/3941200968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8800ab400 msgr2=0x7fc88014abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.503+0000 7fc88da14640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8800ab400 0x7fc88014abb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc88804f7d0 tx=0x7fc888070850 comp rx=0 tx=0).stop 2024-04-25T00:03:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.504+0000 7fc88da14640 1 -- 172.21.15.110:0/3941200968 shutdown_connections 2024-04-25T00:03:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.504+0000 7fc88da14640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8780a2d60 0x7fc8780a5220 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.504+0000 7fc88da14640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8800ab400 0x7fc88014abb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.504+0000 7fc88da14640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8800a6700 0x7fc880143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.504+0000 7fc88da14640 1 --2- 172.21.15.110:0/3941200968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc880018290 0x7fc880143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.504+0000 7fc88da14640 1 -- 172.21.15.110:0/3941200968 >> 172.21.15.110:0/3941200968 conn(0x7fc8800a0520 msgr2=0x7fc8800a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.504+0000 7fc88da14640 1 -- 172.21.15.110:0/3941200968 shutdown_connections 2024-04-25T00:03:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:39.504+0000 7fc88da14640 1 -- 172.21.15.110:0/3941200968 wait complete. 2024-04-25T00:03:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:39 smithi188 ceph-mon[33285]: pgmap v399: 97 pgs: 97 active+clean; 947 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 2.13k op/s 2024-04-25T00:03:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:39 smithi110 ceph-mon[29187]: pgmap v399: 97 pgs: 97 active+clean; 947 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 2.13k op/s 2024-04-25T00:03:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:39 smithi173 ceph-mon[32402]: pgmap v399: 97 pgs: 97 active+clean; 947 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 2.13k op/s 2024-04-25T00:03:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 35.0 s, 1430.8 tps, lat 31.461 ms stddev 14.951 2024-04-25T00:03:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:41 smithi188 ceph-mon[33285]: pgmap v400: 97 pgs: 97 active+clean; 949 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.79k op/s 2024-04-25T00:03:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:41 smithi110 ceph-mon[29187]: pgmap v400: 97 pgs: 97 active+clean; 949 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.79k op/s 2024-04-25T00:03:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:41 smithi173 ceph-mon[32402]: pgmap v400: 97 pgs: 97 active+clean; 949 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.79k op/s 2024-04-25T00:03:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:43 smithi188 ceph-mon[33285]: pgmap v401: 97 pgs: 97 active+clean; 949 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.81k op/s 2024-04-25T00:03:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:43 smithi110 ceph-mon[29187]: pgmap v401: 97 pgs: 97 active+clean; 949 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.81k op/s 2024-04-25T00:03:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:43 smithi173 ceph-mon[32402]: pgmap v401: 97 pgs: 97 active+clean; 949 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.81k op/s 2024-04-25T00:03:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:45 smithi188 ceph-mon[33285]: pgmap v402: 97 pgs: 97 active+clean; 950 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 2.02k op/s 2024-04-25T00:03:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:45 smithi110 ceph-mon[29187]: pgmap v402: 97 pgs: 97 active+clean; 950 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 2.02k op/s 2024-04-25T00:03:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:45 smithi173 ceph-mon[32402]: pgmap v402: 97 pgs: 97 active+clean; 950 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 2.02k op/s 2024-04-25T00:03:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 40.0 s, 1581.0 tps, lat 105.890 ms stddev 530.220 2024-04-25T00:03:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:47 smithi188 ceph-mon[33285]: pgmap v403: 97 pgs: 97 active+clean; 952 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-04-25T00:03:48.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:47 smithi110 ceph-mon[29187]: pgmap v403: 97 pgs: 97 active+clean; 952 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-04-25T00:03:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:47 smithi173 ceph-mon[32402]: pgmap v403: 97 pgs: 97 active+clean; 952 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-04-25T00:03:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:49 smithi188 ceph-mon[33285]: pgmap v404: 97 pgs: 97 active+clean; 952 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 2.08k op/s 2024-04-25T00:03:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:49 smithi110 ceph-mon[29187]: pgmap v404: 97 pgs: 97 active+clean; 952 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 2.08k op/s 2024-04-25T00:03:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:49 smithi173 ceph-mon[32402]: pgmap v404: 97 pgs: 97 active+clean; 952 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 2.08k op/s 2024-04-25T00:03:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 45.0 s, 1434.2 tps, lat 69.652 ms stddev 68.573 2024-04-25T00:03:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:51 smithi188 ceph-mon[33285]: pgmap v405: 97 pgs: 97 active+clean; 956 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-04-25T00:03:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:51 smithi110 ceph-mon[29187]: pgmap v405: 97 pgs: 97 active+clean; 956 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-04-25T00:03:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:51 smithi173 ceph-mon[32402]: pgmap v405: 97 pgs: 97 active+clean; 956 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.645+0000 7fd586e2a640 1 -- 172.21.15.110:0/2163909841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 msgr2=0x7fd58010be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.645+0000 7fd586e2a640 1 --2- 172.21.15.110:0/2163909841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 0x7fd58010be00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd578009ca0 tx=0x7fd57802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.646+0000 7fd586e2a640 1 -- 172.21.15.110:0/2163909841 shutdown_connections 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.646+0000 7fd586e2a640 1 --2- 172.21.15.110:0/2163909841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 0x7fd58010be00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.646+0000 7fd586e2a640 1 --2- 172.21.15.110:0/2163909841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd580104bb0 0x7fd580105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.646+0000 7fd586e2a640 1 --2- 172.21.15.110:0/2163909841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5801039c0 0x7fd580103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.646+0000 7fd586e2a640 1 -- 172.21.15.110:0/2163909841 >> 172.21.15.110:0/2163909841 conn(0x7fd5800fef30 msgr2=0x7fd580101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.646+0000 7fd586e2a640 1 -- 172.21.15.110:0/2163909841 shutdown_connections 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.646+0000 7fd586e2a640 1 -- 172.21.15.110:0/2163909841 wait complete. 2024-04-25T00:03:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd586e2a640 1 Processor -- start 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd586e2a640 1 -- start start 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd586e2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5801039c0 0x7fd580199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd586e2a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd580104bb0 0x7fd58019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd586e2a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 0x7fd5801a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd586e2a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd58010f4f0 con 0x7fd580105570 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd586e2a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd58010f370 con 0x7fd580104bb0 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd586e2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd58010f670 con 0x7fd5801039c0 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd584bc6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5801039c0 0x7fd580199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd584bc6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5801039c0 0x7fd580199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38202/0 (socket says 172.21.15.110:38202) 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd584bc6640 1 -- 172.21.15.110:0/401970534 learned_addr learned my addr 172.21.15.110:0/401970534 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd577fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd580104bb0 0x7fd58019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.647+0000 7fd5853c7640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 0x7fd5801a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.648+0000 7fd584bc6640 1 -- 172.21.15.110:0/401970534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd580104bb0 msgr2=0x7fd58019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.648+0000 7fd584bc6640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd580104bb0 0x7fd58019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.648+0000 7fd584bc6640 1 -- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 msgr2=0x7fd5801a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.648+0000 7fd584bc6640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 0x7fd5801a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.648+0000 7fd584bc6640 1 -- 172.21.15.110:0/401970534 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5801a1700 con 0x7fd5801039c0 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.648+0000 7fd577fff640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd580104bb0 0x7fd58019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.648+0000 7fd5853c7640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 0x7fd5801a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.649+0000 7fd584bc6640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5801039c0 0x7fd580199ae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00e9c0 tx=0x7fd57c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.649+0000 7fd575ffb640 1 -- 172.21.15.110:0/401970534 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00cd80 con 0x7fd5801039c0 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.649+0000 7fd575ffb640 1 -- 172.21.15.110:0/401970534 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd57c0035f0 con 0x7fd5801039c0 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.649+0000 7fd575ffb640 1 -- 172.21.15.110:0/401970534 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c010690 con 0x7fd5801039c0 2024-04-25T00:03:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.649+0000 7fd586e2a640 1 -- 172.21.15.110:0/401970534 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5801a19f0 con 0x7fd5801039c0 2024-04-25T00:03:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.650+0000 7fd586e2a640 1 -- 172.21.15.110:0/401970534 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5801a1ef0 con 0x7fd5801039c0 2024-04-25T00:03:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd575ffb640 1 -- 172.21.15.110:0/401970534 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd57c0037a0 con 0x7fd5801039c0 2024-04-25T00:03:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd589c11740 1 Processor -- start 2024-04-25T00:03:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd589c11740 1 -- start start 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd589c11740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d12ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd589c11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12f050 0x56222d133b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd589c11740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d136650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd589c11740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56222d089cb0 con 0x56222d12e660 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd589c11740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56222d0325b0 con 0x56222d1341c0 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd575ffb640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5680a2cb0 0x7fd5680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.653+0000 7fd589c11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56222d085c70 con 0x56222d12f050 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd584bc6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12f050 0x56222d133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd577fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd577fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d136650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:33234/0 (socket says 172.21.15.110:33234) 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd577fff640 1 -- 172.21.15.110:0/1795681784 learned_addr learned my addr 172.21.15.110:0/1795681784 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd5853c7640 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d12ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd5853c7640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5680a2cb0 0x7fd5680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd575ffb640 1 -- 172.21.15.110:0/401970534 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd57c0cf610 con 0x7fd5801039c0 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd584bc6640 1 -- 172.21.15.110:0/1795681784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 msgr2=0x56222d136650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd584bc6640 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d136650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd584bc6640 1 -- 172.21.15.110:0/1795681784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 msgr2=0x56222d12ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd584bc6640 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d12ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd584bc6640 1 -- 172.21.15.110:0/1795681784 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56222cd9e210 con 0x56222d12f050 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd577fff640 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd584bc6640 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12f050 0x56222d133b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd57c09be30 tx=0x7fd57c01bb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c28eb7bc4549a866 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd5853c7640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5680a2cb0 0x7fd5680a5170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd578009a60 tx=0x7fd578009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.654+0000 7fd566ffd640 1 -- 172.21.15.110:0/1795681784 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c09b800 con 0x56222d12f050 2024-04-25T00:03:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd566ffd640 1 -- 172.21.15.110:0/1795681784 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd57c003b10 con 0x56222d12f050 2024-04-25T00:03:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd566ffd640 1 -- 172.21.15.110:0/1795681784 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0a13e0 con 0x56222d12f050 2024-04-25T00:03:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd589c11740 1 -- 172.21.15.110:0/1795681784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12f050 msgr2=0x56222d133b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd589c11740 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12f050 0x56222d133b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd57c09be30 tx=0x7fd57c01bb30 comp rx=0 tx=0).stop 2024-04-25T00:03:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd589c11740 1 -- 172.21.15.110:0/1795681784 shutdown_connections 2024-04-25T00:03:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd589c11740 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd589c11740 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12f050 0x56222d133b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd589c11740 1 --2- 172.21.15.110:0/1795681784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d12ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.655+0000 7fd589c11740 1 -- 172.21.15.110:0/1795681784 >> 172.21.15.110:0/1795681784 conn(0x56222d129cd0 msgr2=0x56222d12a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.656+0000 7fd589c11740 1 -- 172.21.15.110:0/1795681784 shutdown_connections 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.656+0000 7fd589c11740 1 -- 172.21.15.110:0/1795681784 wait complete. 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.656+0000 7fd589c11740 1 -- start start 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.656+0000 7fd589c11740 1 Processor -- start 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd589c11740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d138a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd589c11740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d12f050 0x56222d138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd589c11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d1341c0 0x56222d140000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd589c11740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56222d137170 con 0x56222d12e660 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd589c11740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56222d045b50 con 0x56222d12f050 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd589c11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56222d1372f0 con 0x56222d1341c0 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd5853c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d1341c0 0x56222d140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd584bc6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d138a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd5853c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d1341c0 0x56222d140000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38224/0 (socket says 172.21.15.110:38224) 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd5853c7640 1 -- 172.21.15.110:0/817327025 learned_addr learned my addr 172.21.15.110:0/817327025 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd577fff640 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d12f050 0x56222d138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd589c11740 10 client.0 _ll_register_callbacks cb 0x56222d080ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd584bc6640 1 -- 172.21.15.110:0/817327025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d1341c0 msgr2=0x56222d140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd584bc6640 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d1341c0 0x56222d140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd584bc6640 1 -- 172.21.15.110:0/817327025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d12f050 msgr2=0x56222d138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd584bc6640 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d12f050 0x56222d138fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.657+0000 7fd584bc6640 1 -- 172.21.15.110:0/817327025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222ce6a620 con 0x56222d12e660 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd589c11740 1 -- 172.21.15.110:0/817327025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56222d0971f0 con 0x56222d12e660 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd584bc6640 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d138a80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0cd190 tx=0x7fd57c0a0250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd198c2f7c224783 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd55ffff640 10 client.15408 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:03:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd55ffff640 1 -- 172.21.15.110:0/817327025 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c09b800 con 0x56222d12e660 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd55ffff640 1 -- 172.21.15.110:0/817327025 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd57c01fe20 con 0x56222d12e660 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd55ffff640 1 -- 172.21.15.110:0/817327025 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd57c09c5a0 con 0x56222d12e660 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd589c11740 10 client.15408 fetch_fsmap learned FSMap version 27 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd589c11740 1 -- 172.21.15.110:0/817327025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=27}) -- 0x56222d143210 con 0x56222d12e660 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.658+0000 7fd55ffff640 1 -- 172.21.15.110:0/817327025 <== mon.0 v2:172.21.15.110:3300/0 4 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fd57c09c100 con 0x56222d12e660 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.659+0000 7fd589c11740 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.659+0000 7fd589c11740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.659+0000 7fd589c11740 20 client.15408 populate_metadata read hostname 'smithi110' 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.659+0000 7fd589c11740 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56222d144a60 0x56222d146f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.659+0000 7fd589c11740 4 client.15408 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.659+0000 7fd589c11740 1 -- 172.21.15.110:0/817327025 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56222d091540 con 0x56222d144a60 2024-04-25T00:03:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.659+0000 7fd577fff640 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56222d144a60 0x56222d146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.660+0000 7fd577fff640 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56222d144a60 0x56222d146f00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a6ee564e3715af6 server_cookie=722fadade7a9fbab in_seq=0 out_seq=0 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.660+0000 7fd55ffff640 10 client.15408 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd55ffff640 1 -- 172.21.15.110:0/817327025 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56222d091540 con 0x56222d144a60 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd55ffff640 10 client.15408 handle_command_reply: tid=0 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 client.15408 shutdown 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 -- 172.21.15.110:0/817327025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 msgr2=0x56222d138a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d138a80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0cd190 tx=0x7fd57c0a0250 comp rx=0 tx=0).stop 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 -- 172.21.15.110:0/817327025 shutdown_connections 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56222d144a60 0x56222d146f00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d1341c0 0x56222d140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d12f050 0x56222d138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 --2- 172.21.15.110:0/817327025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12e660 0x56222d138a80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd589c11740 1 -- 172.21.15.110:0/817327025 >> 172.21.15.110:0/817327025 conn(0x56222d129cd0 msgr2=0x56222d1205c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.661+0000 7fd55ffff640 0 client.15408 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.662+0000 7fd589c11740 1 -- 172.21.15.110:0/817327025 shutdown_connections 2024-04-25T00:03:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.662+0000 7fd589c11740 1 -- 172.21.15.110:0/817327025 wait complete. 2024-04-25T00:03:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.662+0000 7fd589c11740 20 client.15408 trim_cache size 0 max 16384 2024-04-25T00:03:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.689+0000 7fd589c11740 1 Processor -- start 2024-04-25T00:03:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.689+0000 7fd589c11740 1 -- start start 2024-04-25T00:03:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.689+0000 7fd589c11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d138f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.689+0000 7fd589c11740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.689+0000 7fd589c11740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d12d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd589c11740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd5801a90c0 con 0x56222d12f050 2024-04-25T00:03:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd589c11740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd5801a6900 con 0x56222d1341c0 2024-04-25T00:03:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd589c11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5801a4750 con 0x56222d12e660 2024-04-25T00:03:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd584bc6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd584bc6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d138f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38232/0 (socket says 172.21.15.110:38232) 2024-04-25T00:03:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd584bc6640 1 -- 172.21.15.110:0/565615153 learned_addr learned my addr 172.21.15.110:0/565615153 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd577fff640 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd5853c7640 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d12d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd584bc6640 1 -- 172.21.15.110:0/565615153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 msgr2=0x56222d12d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd584bc6640 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d12d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd584bc6640 1 -- 172.21.15.110:0/565615153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 msgr2=0x56222d13ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd584bc6640 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd584bc6640 1 -- 172.21.15.110:0/565615153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56222d12aa40 con 0x56222d12e660 2024-04-25T00:03:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd5853c7640 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d12d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:03:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.690+0000 7fd577fff640 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.691+0000 7fd584bc6640 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d138f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd57c098300 tx=0x7fd57c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34feabca2386c00c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.693+0000 7fd5667fc640 1 -- 172.21.15.110:0/565615153 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c09bea0 con 0x56222d12e660 2024-04-25T00:03:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.693+0000 7fd5667fc640 1 -- 172.21.15.110:0/565615153 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56222d0971f0 con 0x56222d12e660 2024-04-25T00:03:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.693+0000 7fd5667fc640 1 -- 172.21.15.110:0/565615153 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0baae0 con 0x56222d12e660 2024-04-25T00:03:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.694+0000 7fd589c11740 1 -- 172.21.15.110:0/565615153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 msgr2=0x56222d138f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.694+0000 7fd589c11740 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d138f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd57c098300 tx=0x7fd57c011000 comp rx=0 tx=0).stop 2024-04-25T00:03:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.695+0000 7fd589c11740 1 -- 172.21.15.110:0/565615153 shutdown_connections 2024-04-25T00:03:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.695+0000 7fd589c11740 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d12d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.695+0000 7fd589c11740 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.695+0000 7fd589c11740 1 --2- 172.21.15.110:0/565615153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d138f20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.695+0000 7fd589c11740 1 -- 172.21.15.110:0/565615153 >> 172.21.15.110:0/565615153 conn(0x56222d129cd0 msgr2=0x56222d12cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.695+0000 7fd589c11740 1 -- 172.21.15.110:0/565615153 shutdown_connections 2024-04-25T00:03:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.695+0000 7fd589c11740 1 -- 172.21.15.110:0/565615153 wait complete. 2024-04-25T00:03:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.696+0000 7fd589c11740 1 -- start start 2024-04-25T00:03:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.696+0000 7fd589c11740 1 Processor -- start 2024-04-25T00:03:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd589c11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d13d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd589c11740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd589c11740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d13e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd589c11740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd5801a4dc0 con 0x56222d12f050 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd584bc6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d13d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd584bc6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d13d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38242/0 (socket says 172.21.15.110:38242) 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd584bc6640 1 -- 172.21.15.110:0/656417027 learned_addr learned my addr 172.21.15.110:0/656417027 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd577fff640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd5801a6230 con 0x56222d1341c0 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5801a5490 con 0x56222d12e660 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.697+0000 7fd5853c7640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd589c11740 10 client.0 _ll_register_callbacks cb 0x7fd57c023f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:03:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd584bc6640 1 -- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 msgr2=0x56222d13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd584bc6640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d13e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd584bc6640 1 -- 172.21.15.110:0/656417027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 msgr2=0x56222d13d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd584bc6640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd584bc6640 1 -- 172.21.15.110:0/656417027 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222d13ef80 con 0x56222d12e660 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd5853c7640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d13e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd577fff640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd584bc6640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d13d440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0058c0 tx=0x7fd57c0a49b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b9da0ac84cb121e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.698+0000 7fd55ffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.699+0000 7fd55ffff640 1 -- 172.21.15.110:0/656417027 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0a1670 con 0x56222d12e660 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.699+0000 7fd55ffff640 1 -- 172.21.15.110:0/656417027 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd57c09b500 con 0x56222d12e660 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.699+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56222d0971f0 con 0x56222d12e660 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.699+0000 7fd55ffff640 1 -- 172.21.15.110:0/656417027 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd57c09bea0 con 0x56222d12e660 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.699+0000 7fd589c11740 10 client.24973 fetch_fsmap learned FSMap version 27 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.699+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56222d13ef80 con 0x56222d12e660 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.700+0000 7fd55ffff640 1 -- 172.21.15.110:0/656417027 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd57c097bf0 con 0x56222d12e660 2024-04-25T00:03:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.700+0000 7fd55ffff640 1 -- 172.21.15.110:0/656417027 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fd57c0cf070 con 0x56222d12e660 2024-04-25T00:03:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.700+0000 7fd589c11740 10 client.24973 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:03:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.700+0000 7fd589c11740 10 client.24973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:03:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.701+0000 7fd589c11740 20 client.24973 populate_metadata read hostname 'smithi110' 2024-04-25T00:03:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.701+0000 7fd589c11740 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56222d1ab380 0x56222d1ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:03:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.701+0000 7fd589c11740 4 client.24973 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:03:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.701+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56222d0971f0 con 0x56222d1ab380 2024-04-25T00:03:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.701+0000 7fd577fff640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56222d1ab380 0x56222d1ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:03:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.702+0000 7fd577fff640 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56222d1ab380 0x56222d1ad7a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f868969fd3533b31 server_cookie=428fed3b64010eed in_seq=0 out_seq=0 2024-04-25T00:03:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.702+0000 7fd55ffff640 10 client.24973 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.713+0000 7fd55ffff640 1 -- 172.21.15.110:0/656417027 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56222d0971f0 con 0x56222d1ab380 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.713+0000 7fd55ffff640 10 client.24973 handle_command_reply: tid=0 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.713+0000 7fd589c11740 1 client.24973 shutdown 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.713+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 msgr2=0x56222d13d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.713+0000 7fd589c11740 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d13d440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0058c0 tx=0x7fd57c0a49b0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.713+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 shutdown_connections 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd589c11740 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56222d1ab380 0x56222d1ad7a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd589c11740 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56222d1341c0 0x56222d13e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd589c11740 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56222d12f050 0x56222d13d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd55ffff640 0 client.24973 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd589c11740 1 --2- 172.21.15.110:0/656417027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56222d12e660 0x56222d13d440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 >> 172.21.15.110:0/656417027 conn(0x56222d129cd0 msgr2=0x56222d1205c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 shutdown_connections 2024-04-25T00:03:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd589c11740 1 -- 172.21.15.110:0/656417027 wait complete. 2024-04-25T00:03:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.714+0000 7fd589c11740 20 client.24973 trim_cache size 0 max 16384 2024-04-25T00:03:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:03:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:03:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:03:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:03:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.715+0000 7fd586e2a640 1 -- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5680a2cb0 msgr2=0x7fd5680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.715+0000 7fd586e2a640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5680a2cb0 0x7fd5680a5170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd578009a60 tx=0x7fd578009360 comp rx=0 tx=0).stop 2024-04-25T00:03:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.715+0000 7fd586e2a640 1 -- 172.21.15.110:0/401970534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5801039c0 msgr2=0x7fd580199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:03:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.715+0000 7fd586e2a640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5801039c0 0x7fd580199ae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00e9c0 tx=0x7fd57c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:03:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.716+0000 7fd586e2a640 1 -- 172.21.15.110:0/401970534 shutdown_connections 2024-04-25T00:03:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.716+0000 7fd586e2a640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5680a2cb0 0x7fd5680a5170 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.716+0000 7fd586e2a640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd580105570 0x7fd5801a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.716+0000 7fd586e2a640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd580104bb0 0x7fd58019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.716+0000 7fd586e2a640 1 --2- 172.21.15.110:0/401970534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5801039c0 0x7fd580199ae0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:03:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.716+0000 7fd586e2a640 1 -- 172.21.15.110:0/401970534 >> 172.21.15.110:0/401970534 conn(0x7fd5800fef30 msgr2=0x7fd580107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:03:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.716+0000 7fd586e2a640 1 -- 172.21.15.110:0/401970534 shutdown_connections 2024-04-25T00:03:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:03:52.716+0000 7fd586e2a640 1 -- 172.21.15.110:0/401970534 wait complete. 2024-04-25T00:03:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:53 smithi188 ceph-mon[33285]: pgmap v406: 97 pgs: 97 active+clean; 960 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 1.79k op/s 2024-04-25T00:03:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:53 smithi110 ceph-mon[29187]: pgmap v406: 97 pgs: 97 active+clean; 960 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 1.79k op/s 2024-04-25T00:03:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:53 smithi173 ceph-mon[32402]: pgmap v406: 97 pgs: 97 active+clean; 960 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 1.79k op/s 2024-04-25T00:03:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:55 smithi188 ceph-mon[33285]: pgmap v407: 97 pgs: 97 active+clean; 968 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.93k op/s 2024-04-25T00:03:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:55 smithi110 ceph-mon[29187]: pgmap v407: 97 pgs: 97 active+clean; 968 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.93k op/s 2024-04-25T00:03:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:55 smithi173 ceph-mon[32402]: pgmap v407: 97 pgs: 97 active+clean; 968 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.93k op/s 2024-04-25T00:03:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 50.0 s, 1564.0 tps, lat 63.926 ms stddev 31.212 2024-04-25T00:03:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:57 smithi188 ceph-mon[33285]: pgmap v408: 97 pgs: 97 active+clean; 968 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-04-25T00:03:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:57 smithi110 ceph-mon[29187]: pgmap v408: 97 pgs: 97 active+clean; 968 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-04-25T00:03:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:57 smithi173 ceph-mon[32402]: pgmap v408: 97 pgs: 97 active+clean; 968 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-04-25T00:04:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:03:59 smithi188 ceph-mon[33285]: pgmap v409: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 2.03k op/s 2024-04-25T00:04:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:03:59 smithi110 ceph-mon[29187]: pgmap v409: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 2.03k op/s 2024-04-25T00:04:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:03:59 smithi173 ceph-mon[32402]: pgmap v409: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 2.03k op/s 2024-04-25T00:04:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 55.0 s, 1719.0 tps, lat 58.215 ms stddev 14.831 2024-04-25T00:04:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:01 smithi188 ceph-mon[33285]: pgmap v410: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 1.70k op/s 2024-04-25T00:04:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:01 smithi110 ceph-mon[29187]: pgmap v410: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 1.70k op/s 2024-04-25T00:04:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:01 smithi173 ceph-mon[32402]: pgmap v410: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 1.70k op/s 2024-04-25T00:04:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:03 smithi188 ceph-mon[33285]: pgmap v411: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 1.89k op/s 2024-04-25T00:04:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:03 smithi110 ceph-mon[29187]: pgmap v411: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 1.89k op/s 2024-04-25T00:04:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:03 smithi173 ceph-mon[32402]: pgmap v411: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 1.89k op/s 2024-04-25T00:04:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 -- 172.21.15.110:0/1894243755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 msgr2=0x7f2714110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 --2- 172.21.15.110:0/1894243755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 0x7f2714110210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f270c009be0 tx=0x7f270c02a6c0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 -- 172.21.15.110:0/1894243755 shutdown_connections 2024-04-25T00:04:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 --2- 172.21.15.110:0/1894243755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 0x7f2714110210 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 --2- 172.21.15.110:0/1894243755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2714108fc0 0x7f2714109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 --2- 172.21.15.110:0/1894243755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714107dd0 0x7f27141081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 -- 172.21.15.110:0/1894243755 >> 172.21.15.110:0/1894243755 conn(0x7f2714076d90 msgr2=0x7f27140791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 -- 172.21.15.110:0/1894243755 shutdown_connections 2024-04-25T00:04:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.860+0000 7f271be6d640 1 -- 172.21.15.110:0/1894243755 wait complete. 2024-04-25T00:04:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.861+0000 7f271be6d640 1 Processor -- start 2024-04-25T00:04:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.861+0000 7f271be6d640 1 -- start start 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.861+0000 7f271be6d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714107dd0 0x7f271419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.861+0000 7f271be6d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2714108fc0 0x7f271419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271be6d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 0x7f27141a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271be6d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27141138e0 con 0x7f2714108fc0 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271be6d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2714113760 con 0x7f2714109980 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271be6d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2714113a60 con 0x7f2714107dd0 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f2719c09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714107dd0 0x7f271419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271a40a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 0x7f27141a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271a40a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 0x7f27141a5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43992/0 (socket says 172.21.15.110:43992) 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271a40a640 1 -- 172.21.15.110:0/647453311 learned_addr learned my addr 172.21.15.110:0/647453311 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f2719408640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2714108fc0 0x7f271419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271a40a640 1 -- 172.21.15.110:0/647453311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714107dd0 msgr2=0x7f271419dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271a40a640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714107dd0 0x7f271419dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271a40a640 1 -- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2714108fc0 msgr2=0x7f271419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271a40a640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2714108fc0 0x7f271419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.862+0000 7f271a40a640 1 -- 172.21.15.110:0/647453311 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27141a5ae0 con 0x7f2714109980 2024-04-25T00:04:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.863+0000 7f271a40a640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 0x7f27141a5470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f270c02abe0 tx=0x7f270c030f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.863+0000 7f270affd640 1 -- 172.21.15.110:0/647453311 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c051070 con 0x7f2714109980 2024-04-25T00:04:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.863+0000 7f270affd640 1 -- 172.21.15.110:0/647453311 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f270c037040 con 0x7f2714109980 2024-04-25T00:04:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.863+0000 7f270affd640 1 -- 172.21.15.110:0/647453311 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c036070 con 0x7f2714109980 2024-04-25T00:04:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.863+0000 7f271be6d640 1 -- 172.21.15.110:0/647453311 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271406b160 con 0x7f2714109980 2024-04-25T00:04:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.864+0000 7f271be6d640 1 -- 172.21.15.110:0/647453311 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271406b600 con 0x7f2714109980 2024-04-25T00:04:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.867+0000 7f270affd640 1 -- 172.21.15.110:0/647453311 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f270c054070 con 0x7f2714109980 2024-04-25T00:04:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.867+0000 7f271eb77740 1 Processor -- start 2024-04-25T00:04:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.868+0000 7f271eb77740 1 -- start start 2024-04-25T00:04:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.868+0000 7f271eb77740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e19655270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.868+0000 7f271eb77740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19655840 0x555e1965a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f271eb77740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e1965a9b0 0x555e1965ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f271eb77740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555e195b8cb0 con 0x555e19655840 2024-04-25T00:04:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f271eb77740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555e195615b0 con 0x555e1965a9b0 2024-04-25T00:04:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f271eb77740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555e195b4c70 con 0x555e19654e50 2024-04-25T00:04:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f270affd640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27000a2da0 0x7f27000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f2719c09640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19655840 0x555e1965a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f2719c09640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19655840 0x555e1965a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33536/0 (socket says 172.21.15.110:33536) 2024-04-25T00:04:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f2719c09640 1 -- 172.21.15.110:0/1482001799 learned_addr learned my addr 172.21.15.110:0/1482001799 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f2719408640 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e19655270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.869+0000 7f270affd640 1 -- 172.21.15.110:0/647453311 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f270c100780 con 0x7f2714109980 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f2719408640 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e1965a9b0 0x555e1965ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f2719408640 1 -- 172.21.15.110:0/1482001799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e1965a9b0 msgr2=0x555e1965ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f2719408640 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e1965a9b0 0x555e1965ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f2719408640 1 -- 172.21.15.110:0/1482001799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19655840 msgr2=0x555e1965a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f2719408640 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19655840 0x555e1965a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f2719408640 1 -- 172.21.15.110:0/1482001799 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e192cd210 con 0x555e19654e50 2024-04-25T00:04:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f271a40a640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27000a2da0 0x7f27000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f2719c09640 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19655840 0x555e1965a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.870+0000 7f2719408640 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e19655270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f271419f3d0 tx=0x7f2704005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c77704b38fd251e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.871+0000 7f271a40a640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27000a2da0 0x7f27000a5260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f270c0ccfc0 tx=0x7f270c009280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.871+0000 7f26fffff640 1 -- 172.21.15.110:0/1482001799 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704004020 con 0x555e19654e50 2024-04-25T00:04:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.871+0000 7f26fffff640 1 -- 172.21.15.110:0/1482001799 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27040041c0 con 0x555e19654e50 2024-04-25T00:04:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.871+0000 7f26fffff640 1 -- 172.21.15.110:0/1482001799 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270400f880 con 0x555e19654e50 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.872+0000 7f271eb77740 1 -- 172.21.15.110:0/1482001799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 msgr2=0x555e19655270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.872+0000 7f271eb77740 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e19655270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f271419f3d0 tx=0x7f2704005a00 comp rx=0 tx=0).stop 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.872+0000 7f271eb77740 1 -- 172.21.15.110:0/1482001799 shutdown_connections 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.872+0000 7f271eb77740 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e1965a9b0 0x555e1965ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.872+0000 7f271eb77740 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19655840 0x555e1965a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.872+0000 7f271eb77740 1 --2- 172.21.15.110:0/1482001799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e19655270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.872+0000 7f271eb77740 1 -- 172.21.15.110:0/1482001799 >> 172.21.15.110:0/1482001799 conn(0x555e196504c0 msgr2=0x555e196508c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.873+0000 7f271eb77740 1 -- 172.21.15.110:0/1482001799 shutdown_connections 2024-04-25T00:04:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.873+0000 7f271eb77740 1 -- 172.21.15.110:0/1482001799 wait complete. 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.873+0000 7f271eb77740 1 -- start start 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.873+0000 7f271eb77740 1 Processor -- start 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f271eb77740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f271eb77740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e1965f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f271eb77740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 0x555e19666850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f2719408640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f2719408640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:51760/0 (socket says 172.21.15.110:51760) 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f2719408640 1 -- 172.21.15.110:0/678182273 learned_addr learned my addr 172.21.15.110:0/678182273 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555e1965d900 con 0x555e1965a9b0 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555e19574b50 con 0x555e19655840 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555e1965da80 con 0x555e19654e50 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f271eb77740 10 client.0 _ll_register_callbacks cb 0x555e195afca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f2719c09640 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e1965f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.874+0000 7f2719408640 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 0x555e19666850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f2719408640 1 -- 172.21.15.110:0/678182273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 msgr2=0x555e1965f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f2719408640 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e1965f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f2719408640 1 -- 172.21.15.110:0/678182273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 msgr2=0x555e19666850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f2719408640 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 0x555e19666850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f2719408640 1 -- 172.21.15.110:0/678182273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e19399620 con 0x555e19654e50 2024-04-25T00:04:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f2719408640 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965f2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27040057c0 tx=0x7f2704009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee02af07ac5b34f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f26fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:04:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f26fd7fa640 1 -- 172.21.15.110:0/678182273 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704028040 con 0x555e19654e50 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.875+0000 7f26fd7fa640 1 -- 172.21.15.110:0/678182273 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2704005040 con 0x555e19654e50 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27040041c0 con 0x555e19654e50 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f26fd7fa640 1 -- 172.21.15.110:0/678182273 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2704003b80 con 0x555e19654e50 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 10 client.24985 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555e19669ba0 con 0x555e19654e50 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f26fd7fa640 1 -- 172.21.15.110:0/678182273 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27040041c0 con 0x555e19654e50 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f26fd7fa640 1 -- 172.21.15.110:0/678182273 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f270401d020 con 0x555e19654e50 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 10 client.24985 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 10 client.24985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 20 client.24985 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555e1966b020 0x555e1966d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 4 client.24985 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.876+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e195c61f0 con 0x555e1966b020 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.877+0000 7f2719c09640 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555e1966b020 0x555e1966d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.877+0000 7f2719c09640 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555e1966b020 0x555e1966d410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d9085f8e07fc49 server_cookie=c84b8a440d9a2199 in_seq=0 out_seq=0 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.877+0000 7f26fd7fa640 10 client.24985 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f26fd7fa640 1 -- 172.21.15.110:0/678182273 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e195c61f0 con 0x555e1966b020 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f26fd7fa640 10 client.24985 handle_command_reply: tid=0 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 client.24985 shutdown 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 msgr2=0x555e1965f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965f2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27040057c0 tx=0x7f2704009fb0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 shutdown_connections 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555e1966b020 0x555e1966d410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f26fd7fa640 0 client.24985 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 0x555e19666850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e1965f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 --2- 172.21.15.110:0/678182273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965f2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.878+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 >> 172.21.15.110:0/678182273 conn(0x555e196504c0 msgr2=0x555e1964f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.879+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 shutdown_connections 2024-04-25T00:04:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.879+0000 7f271eb77740 1 -- 172.21.15.110:0/678182273 wait complete. 2024-04-25T00:04:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.879+0000 7f271eb77740 20 client.24985 trim_cache size 0 max 16384 2024-04-25T00:04:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.905+0000 7f271eb77740 1 Processor -- start 2024-04-25T00:04:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.905+0000 7f271eb77740 1 -- start start 2024-04-25T00:04:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f271eb77740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f271eb77740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f271eb77740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 0x555e1965f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f271eb77740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27141a9170 con 0x555e1965a9b0 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f271eb77740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f27141a69b0 con 0x555e19655840 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f271eb77740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27140724b0 con 0x555e19654e50 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:51768/0 (socket says 172.21.15.110:51768) 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 -- 172.21.15.110:0/3452912618 learned_addr learned my addr 172.21.15.110:0/3452912618 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719c09640 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 0x555e1965f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 -- 172.21.15.110:0/3452912618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 msgr2=0x555e196667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 -- 172.21.15.110:0/3452912618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 msgr2=0x555e1965f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 0x555e1965f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.906+0000 7f2719408640 1 -- 172.21.15.110:0/3452912618 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e19651230 con 0x555e19654e50 2024-04-25T00:04:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.907+0000 7f2719408640 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965de80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f270400f7b0 tx=0x7f2704009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5505d4e30fa1106d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.907+0000 7f26ff7fe640 1 -- 172.21.15.110:0/3452912618 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704040070 con 0x555e19654e50 2024-04-25T00:04:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.907+0000 7f26ff7fe640 1 -- 172.21.15.110:0/3452912618 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2704004660 con 0x555e19654e50 2024-04-25T00:04:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.907+0000 7f26ff7fe640 1 -- 172.21.15.110:0/3452912618 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270400fb10 con 0x555e19654e50 2024-04-25T00:04:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 -- 172.21.15.110:0/3452912618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 msgr2=0x555e1965de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965de80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f270400f7b0 tx=0x7f2704009260 comp rx=0 tx=0).stop 2024-04-25T00:04:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 -- 172.21.15.110:0/3452912618 shutdown_connections 2024-04-25T00:04:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e1965a9b0 0x555e1965f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 --2- 172.21.15.110:0/3452912618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e19654e50 0x555e1965de80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 -- 172.21.15.110:0/3452912618 >> 172.21.15.110:0/3452912618 conn(0x555e196504c0 msgr2=0x555e196508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 -- 172.21.15.110:0/3452912618 shutdown_connections 2024-04-25T00:04:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.908+0000 7f271eb77740 1 -- 172.21.15.110:0/3452912618 wait complete. 2024-04-25T00:04:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.909+0000 7f271eb77740 1 -- start start 2024-04-25T00:04:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.909+0000 7f271eb77740 1 Processor -- start 2024-04-25T00:04:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f271eb77740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19654e50 0x555e19663490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f271eb77740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f271eb77740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e1965a9b0 0x555e196cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f271eb77740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2714072b20 con 0x555e19654e50 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f271eb77740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f27141a62e0 con 0x555e19655840 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f271eb77740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27140731f0 con 0x555e1965a9b0 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f2719408640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19654e50 0x555e19663490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f2719408640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19654e50 0x555e19663490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33566/0 (socket says 172.21.15.110:33566) 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f271eb77740 10 client.0 _ll_register_callbacks cb 0x7f27100097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f2719408640 1 -- 172.21.15.110:0/2459943662 learned_addr learned my addr 172.21.15.110:0/2459943662 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.910+0000 7f2719c09640 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f2719408640 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e1965a9b0 0x555e196cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f2719c09640 1 -- 172.21.15.110:0/2459943662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e1965a9b0 msgr2=0x555e196cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f2719c09640 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e1965a9b0 0x555e196cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f2719c09640 1 -- 172.21.15.110:0/2459943662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19654e50 msgr2=0x555e19663490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f2719c09640 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19654e50 0x555e19663490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f2719c09640 1 -- 172.21.15.110:0/2459943662 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e19663fc0 con 0x555e19655840 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f2719c09640 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196639d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x555e1966dc70 tx=0x7f2710020790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=420d5f95efc86c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f26fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.911+0000 7f26fd7fa640 1 -- 172.21.15.110:0/2459943662 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710010d70 con 0x555e19655840 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 1 -- 172.21.15.110:0/2459943662 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2704004660 con 0x555e19655840 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f26fd7fa640 1 -- 172.21.15.110:0/2459943662 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2710005020 con 0x555e19655840 2024-04-25T00:04:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f26fd7fa640 1 -- 172.21.15.110:0/2459943662 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f271002f070 con 0x555e19655840 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 10 client.25013 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 1 -- 172.21.15.110:0/2459943662 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555e196d0bf0 con 0x555e19655840 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f26fd7fa640 1 -- 172.21.15.110:0/2459943662 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2704004660 con 0x555e19655840 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f26fd7fa640 1 -- 172.21.15.110:0/2459943662 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f2710004c40 con 0x555e19655840 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 10 client.25013 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 10 client.25013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 20 client.25013 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555e1966b020 0x555e196d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 4 client.25013 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.912+0000 7f271eb77740 1 -- 172.21.15.110:0/2459943662 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e195c0540 con 0x555e1966b020 2024-04-25T00:04:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.913+0000 7f2719408640 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555e1966b020 0x555e196d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.913+0000 7f26fd7fa640 10 client.25013 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.913+0000 7f2719408640 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555e1966b020 0x555e196d35f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd5290f07a2985a server_cookie=da78560abad3f0c4 in_seq=0 out_seq=0 2024-04-25T00:04:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.923+0000 7f26fd7fa640 1 -- 172.21.15.110:0/2459943662 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e195c0540 con 0x555e1966b020 2024-04-25T00:04:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.923+0000 7f26fd7fa640 10 client.25013 handle_command_reply: tid=0 2024-04-25T00:04:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 client.25013 shutdown 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 -- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 msgr2=0x555e196639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196639d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x555e1966dc70 tx=0x7f2710020790 comp rx=0 tx=0).stop 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 -- 172.21.15.110:0/2459943662 shutdown_connections 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555e1966b020 0x555e196d35f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e1965a9b0 0x555e196cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f26fd7fa640 0 client.25013 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e19655840 0x555e196639d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 --2- 172.21.15.110:0/2459943662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555e19654e50 0x555e19663490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 -- 172.21.15.110:0/2459943662 >> 172.21.15.110:0/2459943662 conn(0x555e196504c0 msgr2=0x555e196508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.924+0000 7f271eb77740 1 -- 172.21.15.110:0/2459943662 shutdown_connections 2024-04-25T00:04:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.925+0000 7f271eb77740 1 -- 172.21.15.110:0/2459943662 wait complete. 2024-04-25T00:04:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.925+0000 7f271eb77740 20 client.25013 trim_cache size 0 max 16384 2024-04-25T00:04:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:04:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:04:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:04:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:04:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 -- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27000a2da0 msgr2=0x7f27000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27000a2da0 0x7f27000a5260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f270c0ccfc0 tx=0x7f270c009280 comp rx=0 tx=0).stop 2024-04-25T00:04:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 -- 172.21.15.110:0/647453311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 msgr2=0x7f27141a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 0x7f27141a5470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f270c02abe0 tx=0x7f270c030f10 comp rx=0 tx=0).stop 2024-04-25T00:04:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 -- 172.21.15.110:0/647453311 shutdown_connections 2024-04-25T00:04:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f27000a2da0 0x7f27000a5260 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2714109980 0x7f27141a5470 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2714108fc0 0x7f271419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 --2- 172.21.15.110:0/647453311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2714107dd0 0x7f271419dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.926+0000 7f271be6d640 1 -- 172.21.15.110:0/647453311 >> 172.21.15.110:0/647453311 conn(0x7f2714076d90 msgr2=0x7f271410c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.927+0000 7f271be6d640 1 -- 172.21.15.110:0/647453311 shutdown_connections 2024-04-25T00:04:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:05.927+0000 7f271be6d640 1 -- 172.21.15.110:0/647453311 wait complete. 2024-04-25T00:04:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:05 smithi188 ceph-mon[33285]: pgmap v412: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 2.02k op/s 2024-04-25T00:04:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:05 smithi110 ceph-mon[29187]: pgmap v412: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 2.02k op/s 2024-04-25T00:04:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:05 smithi173 ceph-mon[32402]: pgmap v412: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 2.02k op/s 2024-04-25T00:04:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 60.0 s, 1715.2 tps, lat 58.261 ms stddev 15.251 2024-04-25T00:04:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:07 smithi188 ceph-mon[33285]: pgmap v413: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-04-25T00:04:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:07 smithi110 ceph-mon[29187]: pgmap v413: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-04-25T00:04:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:07 smithi173 ceph-mon[32402]: pgmap v413: 97 pgs: 97 active+clean; 972 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-04-25T00:04:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:09 smithi188 ceph-mon[33285]: pgmap v414: 97 pgs: 97 active+clean; 972 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 2.25k op/s 2024-04-25T00:04:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:09 smithi110 ceph-mon[29187]: pgmap v414: 97 pgs: 97 active+clean; 972 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 2.25k op/s 2024-04-25T00:04:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:09 smithi173 ceph-mon[32402]: pgmap v414: 97 pgs: 97 active+clean; 972 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 2.25k op/s 2024-04-25T00:04:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 65.0 s, 1469.0 tps, lat 68.163 ms stddev 59.127 2024-04-25T00:04:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:11 smithi188 ceph-mon[33285]: pgmap v415: 97 pgs: 97 active+clean; 972 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.86k op/s 2024-04-25T00:04:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:11 smithi110 ceph-mon[29187]: pgmap v415: 97 pgs: 97 active+clean; 972 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.86k op/s 2024-04-25T00:04:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:11 smithi173 ceph-mon[32402]: pgmap v415: 97 pgs: 97 active+clean; 972 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.86k op/s 2024-04-25T00:04:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:13 smithi188 ceph-mon[33285]: pgmap v416: 97 pgs: 97 active+clean; 976 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.89k op/s 2024-04-25T00:04:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:13 smithi110 ceph-mon[29187]: pgmap v416: 97 pgs: 97 active+clean; 976 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.89k op/s 2024-04-25T00:04:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:13 smithi173 ceph-mon[32402]: pgmap v416: 97 pgs: 97 active+clean; 976 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 1.89k op/s 2024-04-25T00:04:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:15 smithi188 ceph-mon[33285]: pgmap v417: 97 pgs: 97 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.88k op/s 2024-04-25T00:04:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:15 smithi110 ceph-mon[29187]: pgmap v417: 97 pgs: 97 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.88k op/s 2024-04-25T00:04:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:15 smithi173 ceph-mon[32402]: pgmap v417: 97 pgs: 97 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.88k op/s 2024-04-25T00:04:16.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 70.0 s, 1656.0 tps, lat 60.328 ms stddev 16.904 2024-04-25T00:04:18.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:17 smithi188 ceph-mon[33285]: pgmap v418: 97 pgs: 97 active+clean; 988 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 1.66k op/s 2024-04-25T00:04:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:17 smithi110 ceph-mon[29187]: pgmap v418: 97 pgs: 97 active+clean; 988 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 1.66k op/s 2024-04-25T00:04:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:17 smithi173 ceph-mon[32402]: pgmap v418: 97 pgs: 97 active+clean; 988 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 1.66k op/s 2024-04-25T00:04:19.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:18 smithi110 ceph-mon[29187]: pgmap v419: 97 pgs: 97 active+clean; 988 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 2.04k op/s 2024-04-25T00:04:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.070+0000 7f66edc53640 1 -- 172.21.15.110:0/3963954232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66e8102260 msgr2=0x7f66e8104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.070+0000 7f66edc53640 1 --2- 172.21.15.110:0/3963954232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66e8102260 0x7f66e8104650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009900 tx=0x7f66d802a410 comp rx=0 tx=0).stop 2024-04-25T00:04:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.071+0000 7f66edc53640 1 -- 172.21.15.110:0/3963954232 shutdown_connections 2024-04-25T00:04:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.071+0000 7f66edc53640 1 --2- 172.21.15.110:0/3963954232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66e81074c0 0x7f66e81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.071+0000 7f66edc53640 1 --2- 172.21.15.110:0/3963954232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e8104b90 0x7f66e8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.071+0000 7f66edc53640 1 --2- 172.21.15.110:0/3963954232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66e8102260 0x7f66e8104650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.071+0000 7f66edc53640 1 -- 172.21.15.110:0/3963954232 >> 172.21.15.110:0/3963954232 conn(0x7f66e80fbc10 msgr2=0x7f66e80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.071+0000 7f66edc53640 1 -- 172.21.15.110:0/3963954232 shutdown_connections 2024-04-25T00:04:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.071+0000 7f66edc53640 1 -- 172.21.15.110:0/3963954232 wait complete. 2024-04-25T00:04:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.072+0000 7f66edc53640 1 Processor -- start 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.072+0000 7f66edc53640 1 -- start start 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.072+0000 7f66edc53640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e8102260 0x7f66e8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.072+0000 7f66edc53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66e8104b90 0x7f66e8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e8102260 0x7f66e8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e8102260 0x7f66e8197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:33358/0 (socket says 172.21.15.110:33358) 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e77fe640 1 -- 172.21.15.110:0/1271058887 learned_addr learned my addr 172.21.15.110:0/1271058887 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66edc53640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66e81074c0 0x7f66e819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66e8104b90 0x7f66e8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66e810d370 con 0x7f66e81074c0 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66e810d1f0 con 0x7f66e8102260 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f66e810d4f0 con 0x7f66e8104b90 2024-04-25T00:04:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e7fff640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66e81074c0 0x7f66e819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e77fe640 1 -- 172.21.15.110:0/1271058887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66e8104b90 msgr2=0x7f66e8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e77fe640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66e8104b90 0x7f66e8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e77fe640 1 -- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66e81074c0 msgr2=0x7f66e819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e77fe640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66e81074c0 0x7f66e819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.073+0000 7f66e77fe640 1 -- 172.21.15.110:0/1271058887 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e819f5b0 con 0x7f66e8102260 2024-04-25T00:04:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.074+0000 7f66e77fe640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e8102260 0x7f66e8197990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66d80314e0 tx=0x7f66d80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.074+0000 7f66e4ff9640 1 -- 172.21.15.110:0/1271058887 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8035030 con 0x7f66e8102260 2024-04-25T00:04:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.074+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e8100050 con 0x7f66e8102260 2024-04-25T00:04:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.074+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e8100590 con 0x7f66e8102260 2024-04-25T00:04:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.074+0000 7f66e4ff9640 1 -- 172.21.15.110:0/1271058887 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d8041070 con 0x7f66e8102260 2024-04-25T00:04:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.074+0000 7f66e4ff9640 1 -- 172.21.15.110:0/1271058887 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8031bc0 con 0x7f66e8102260 2024-04-25T00:04:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.078+0000 7f66e4ff9640 1 -- 172.21.15.110:0/1271058887 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f66d8046470 con 0x7f66e8102260 2024-04-25T00:04:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.078+0000 7f66e4ff9640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66d00a2c30 0x7f66d00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66e4ff9640 1 -- 172.21.15.110:0/1271058887 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f66d80f3990 con 0x7f66e8102260 2024-04-25T00:04:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66d00a2c30 0x7f66d00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66f0a34740 1 Processor -- start 2024-04-25T00:04:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66f0a34740 1 -- start start 2024-04-25T00:04:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc4fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc54b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0cc57650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66f0a34740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559b0cbaacb0 con 0x559b0cc50050 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66f0a34740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b0cb535b0 con 0x559b0cc4f660 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.079+0000 7f66f0a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559b0cba6c70 con 0x559b0cc551c0 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0cc57650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc4fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc54b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc4fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:33362/0 (socket says 172.21.15.110:33362) 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e77fe640 1 -- 172.21.15.110:0/1682523742 learned_addr learned my addr 172.21.15.110:0/1682523742 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0cc57650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48722/0 (socket says 172.21.15.110:48722) 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66d00a2c30 0x7f66d00a50f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66e8198ea0 tx=0x7f66dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e7fff640 1 -- 172.21.15.110:0/1682523742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 msgr2=0x559b0cc4fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e7fff640 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc4fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e7fff640 1 -- 172.21.15.110:0/1682523742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 msgr2=0x559b0cc54b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e7fff640 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc54b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e7fff640 1 -- 172.21.15.110:0/1682523742 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0c8bf210 con 0x559b0cc551c0 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e77fe640 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc4fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.080+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc54b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.081+0000 7f66e7fff640 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0cc57650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009ed0 tx=0x7f66e00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=139c45a022cd5604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.081+0000 7f66cdffb640 1 -- 172.21.15.110:0/1682523742 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0003b70 con 0x559b0cc551c0 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.081+0000 7f66cdffb640 1 -- 172.21.15.110:0/1682523742 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e0015070 con 0x559b0cc551c0 2024-04-25T00:04:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.081+0000 7f66cdffb640 1 -- 172.21.15.110:0/1682523742 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00117b0 con 0x559b0cc551c0 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.082+0000 7f66f0a34740 1 -- 172.21.15.110:0/1682523742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 msgr2=0x559b0cc57650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.082+0000 7f66f0a34740 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0cc57650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009ed0 tx=0x7f66e00107a0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.082+0000 7f66f0a34740 1 -- 172.21.15.110:0/1682523742 shutdown_connections 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.082+0000 7f66f0a34740 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0cc57650 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.082+0000 7f66f0a34740 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc54b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.082+0000 7f66f0a34740 1 --2- 172.21.15.110:0/1682523742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc4fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.082+0000 7f66f0a34740 1 -- 172.21.15.110:0/1682523742 >> 172.21.15.110:0/1682523742 conn(0x559b0cc424c0 msgr2=0x559b0cc428c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.082+0000 7f66f0a34740 1 -- 172.21.15.110:0/1682523742 shutdown_connections 2024-04-25T00:04:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.083+0000 7f66f0a34740 1 -- 172.21.15.110:0/1682523742 wait complete. 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.083+0000 7f66f0a34740 1 -- start start 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.083+0000 7f66f0a34740 1 Processor -- start 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc61030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66f0a34740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559b0cc58170 con 0x559b0cc551c0 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66f0a34740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b0cb66b50 con 0x559b0cc4f660 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66f0a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559b0cc582f0 con 0x559b0cc50050 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66f0a34740 10 client.0 _ll_register_callbacks cb 0x559b0cba1ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc61030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48726/0 (socket says 172.21.15.110:48726) 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e6ffd640 1 -- 172.21.15.110:0/959890900 learned_addr learned my addr 172.21.15.110:0/959890900 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e6ffd640 1 -- 172.21.15.110:0/959890900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 msgr2=0x559b0cc59ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e6ffd640 1 -- 172.21.15.110:0/959890900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 msgr2=0x559b0cc61030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc61030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e6ffd640 1 -- 172.21.15.110:0/959890900 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0c98b620 con 0x559b0cc50050 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.084+0000 7f66e77fe640 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.085+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66dc01ec60 tx=0x7f66dc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d444e8789ee4468e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.085+0000 7f66c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.085+0000 7f66c6ffd640 1 -- 172.21.15.110:0/959890900 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc00ec00 con 0x559b0cc50050 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.085+0000 7f66f0a34740 1 -- 172.21.15.110:0/959890900 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e0015070 con 0x559b0cc50050 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.085+0000 7f66c6ffd640 1 -- 172.21.15.110:0/959890900 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f66dc00eda0 con 0x559b0cc50050 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.085+0000 7f66c6ffd640 1 -- 172.21.15.110:0/959890900 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66dc005020 con 0x559b0cc50050 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.085+0000 7f66f0a34740 10 client.25003 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.085+0000 7f66f0a34740 1 -- 172.21.15.110:0/959890900 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b0cc644f0 con 0x559b0cc50050 2024-04-25T00:04:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66c6ffd640 1 -- 172.21.15.110:0/959890900 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e0015070 con 0x559b0cc50050 2024-04-25T00:04:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66c6ffd640 1 -- 172.21.15.110:0/959890900 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f66dc025070 con 0x559b0cc50050 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66f0a34740 10 client.25003 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66f0a34740 10 client.25003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66f0a34740 20 client.25003 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66f0a34740 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559b0cc65cf0 0x559b0cc68190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66f0a34740 4 client.25003 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66f0a34740 1 -- 172.21.15.110:0/959890900 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b0cbb81f0 con 0x559b0cc65cf0 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.086+0000 7f66e77fe640 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559b0cc65cf0 0x559b0cc68190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.089+0000 7f66e77fe640 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559b0cc65cf0 0x559b0cc68190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7d83fb8ad27bc6 server_cookie=6415ed1d26ae104a in_seq=0 out_seq=0 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.089+0000 7f66c6ffd640 10 client.25003 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.089+0000 7f66c6ffd640 1 -- 172.21.15.110:0/959890900 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b0cbb81f0 con 0x559b0cc65cf0 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.089+0000 7f66c6ffd640 10 client.25003 handle_command_reply: tid=0 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.089+0000 7f66f0a34740 1 client.25003 shutdown 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 -- 172.21.15.110:0/959890900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc50050 msgr2=0x559b0cc59ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66dc01ec60 tx=0x7f66dc004730 comp rx=0 tx=0).stop 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 -- 172.21.15.110:0/959890900 shutdown_connections 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559b0cc65cf0 0x559b0cc68190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc61030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 --2- 172.21.15.110:0/959890900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66c6ffd640 0 client.25003 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 -- 172.21.15.110:0/959890900 >> 172.21.15.110:0/959890900 conn(0x559b0cc424c0 msgr2=0x559b0cc415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 -- 172.21.15.110:0/959890900 shutdown_connections 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 1 -- 172.21.15.110:0/959890900 wait complete. 2024-04-25T00:04:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.090+0000 7f66f0a34740 20 client.25003 trim_cache size 0 max 16384 2024-04-25T00:04:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:18 smithi173 ceph-mon[32402]: pgmap v419: 97 pgs: 97 active+clean; 988 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 2.04k op/s 2024-04-25T00:04:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.115+0000 7f66f0a34740 1 Processor -- start 2024-04-25T00:04:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.115+0000 7f66f0a34740 1 -- start start 2024-04-25T00:04:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.115+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.115+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.115+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc60ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.115+0000 7f66f0a34740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66d4010d00 con 0x559b0cc551c0 2024-04-25T00:04:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.115+0000 7f66f0a34740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400e540 con 0x559b0cc50050 2024-04-25T00:04:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.115+0000 7f66f0a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400c390 con 0x559b0cc4f660 2024-04-25T00:04:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc60ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc60ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38930/0 (socket says 172.21.15.110:38930) 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48738/0 (socket says 172.21.15.110:48738) 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e7fff640 1 -- 172.21.15.110:0/2647736727 learned_addr learned my addr 172.21.15.110:0/2647736727 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e6ffd640 1 -- 172.21.15.110:0/2647736727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc4f660 msgr2=0x559b0cc59ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/2647736727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e6ffd640 1 -- 172.21.15.110:0/2647736727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 msgr2=0x559b0cc60ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/2647736727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc60ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e6ffd640 1 -- 172.21.15.110:0/2647736727 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0cc43230 con 0x559b0cc50050 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e7fff640 1 --2- 172.21.15.110:0/2647736727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc60ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.116+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/2647736727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66dc013010 tx=0x7f66dc0119f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a9f34ba2a3039b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.117+0000 7f66cd7fa640 1 -- 172.21.15.110:0/2647736727 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc00e780 con 0x559b0cc50050 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.117+0000 7f66cd7fa640 1 -- 172.21.15.110:0/2647736727 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66dc038070 con 0x559b0cc50050 2024-04-25T00:04:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.117+0000 7f66cd7fa640 1 -- 172.21.15.110:0/2647736727 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0047a0 con 0x559b0cc50050 2024-04-25T00:04:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 -- 172.21.15.110:0/2647736727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc50050 msgr2=0x559b0cc59ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 --2- 172.21.15.110:0/2647736727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66dc013010 tx=0x7f66dc0119f0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 -- 172.21.15.110:0/2647736727 shutdown_connections 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 --2- 172.21.15.110:0/2647736727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc551c0 0x559b0cc60ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 --2- 172.21.15.110:0/2647736727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc50050 0x559b0cc59ff0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 --2- 172.21.15.110:0/2647736727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc4f660 0x559b0cc59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 -- 172.21.15.110:0/2647736727 >> 172.21.15.110:0/2647736727 conn(0x559b0cc424c0 msgr2=0x559b0cc67d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 -- 172.21.15.110:0/2647736727 shutdown_connections 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.118+0000 7f66f0a34740 1 -- 172.21.15.110:0/2647736727 wait complete. 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.119+0000 7f66f0a34740 1 -- start start 2024-04-25T00:04:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.119+0000 7f66f0a34740 1 Processor -- start 2024-04-25T00:04:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.119+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc5ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.119+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.119+0000 7f66f0a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0ccca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.119+0000 7f66f0a34740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400ca00 con 0x559b0cc50050 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.119+0000 7f66f0a34740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400de70 con 0x559b0cc4f660 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.119+0000 7f66f0a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f66d400d0d0 con 0x559b0cc551c0 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66f0a34740 10 client.0 _ll_register_callbacks cb 0x7f66dc0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc5e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38934/0 (socket says 172.21.15.110:38934) 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 -- 172.21.15.110:0/4251561413 learned_addr learned my addr 172.21.15.110:0/4251561413 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e7fff640 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0ccca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 -- 172.21.15.110:0/4251561413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 msgr2=0x559b0ccca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e77fe640 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc5ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0ccca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 -- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 msgr2=0x559b0cc5ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc5ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 -- 172.21.15.110:0/4251561413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0cc5e940 con 0x559b0cc50050 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e77fe640 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc5ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66e6ffd640 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc5e300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009710 tx=0x7f66dc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c7a162e94c0fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.120+0000 7f66c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.121+0000 7f66c6ffd640 1 -- 172.21.15.110:0/4251561413 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc043070 con 0x559b0cc50050 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.121+0000 7f66f0a34740 1 -- 172.21.15.110:0/4251561413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66dc038070 con 0x559b0cc50050 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.121+0000 7f66c6ffd640 1 -- 172.21.15.110:0/4251561413 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f66dc0053d0 con 0x559b0cc50050 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.121+0000 7f66c6ffd640 1 -- 172.21.15.110:0/4251561413 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66dc0055d0 con 0x559b0cc50050 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.121+0000 7f66f0a34740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.121+0000 7f66f0a34740 1 -- 172.21.15.110:0/4251561413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b0cccb900 con 0x559b0cc50050 2024-04-25T00:04:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.122+0000 7f66c6ffd640 1 -- 172.21.15.110:0/4251561413 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66dc038070 con 0x559b0cc50050 2024-04-25T00:04:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.122+0000 7f66c6ffd640 1 -- 172.21.15.110:0/4251561413 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f66dc025050 con 0x559b0cc50050 2024-04-25T00:04:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.122+0000 7f66f0a34740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.122+0000 7f66f0a34740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.122+0000 7f66f0a34740 20 client.15450 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.122+0000 7f66f0a34740 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559b0cccc2e0 0x559b0ccce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.122+0000 7f66f0a34740 4 client.15450 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:04:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.122+0000 7f66f0a34740 1 -- 172.21.15.110:0/4251561413 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b0cbb2540 con 0x559b0cccc2e0 2024-04-25T00:04:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.123+0000 7f66e77fe640 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559b0cccc2e0 0x559b0ccce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.123+0000 7f66c6ffd640 10 client.15450 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.123+0000 7f66e77fe640 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559b0cccc2e0 0x559b0ccce700 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9beaa4b82450a9 server_cookie=ef0bba049997da23 in_seq=0 out_seq=0 2024-04-25T00:04:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66c6ffd640 1 -- 172.21.15.110:0/4251561413 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b0cbb2540 con 0x559b0cccc2e0 2024-04-25T00:04:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66c6ffd640 10 client.15450 handle_command_reply: tid=0 2024-04-25T00:04:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 client.15450 shutdown 2024-04-25T00:04:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 -- 172.21.15.110:0/4251561413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 msgr2=0x559b0cc5e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc5e300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009710 tx=0x7f66dc00ee30 comp rx=0 tx=0).stop 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 -- 172.21.15.110:0/4251561413 shutdown_connections 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559b0cccc2e0 0x559b0ccce700 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559b0cc551c0 0x559b0ccca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b0cc50050 0x559b0cc5e300 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 --2- 172.21.15.110:0/4251561413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b0cc4f660 0x559b0cc5ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.132+0000 7f66f0a34740 1 -- 172.21.15.110:0/4251561413 >> 172.21.15.110:0/4251561413 conn(0x559b0cc424c0 msgr2=0x559b0cc59ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.133+0000 7f66c6ffd640 0 client.15450 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.133+0000 7f66f0a34740 1 -- 172.21.15.110:0/4251561413 shutdown_connections 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.133+0000 7f66f0a34740 1 -- 172.21.15.110:0/4251561413 wait complete. 2024-04-25T00:04:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:04:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:04:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:04:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.133+0000 7f66f0a34740 20 client.15450 trim_cache size 0 max 16384 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66d00a2c30 msgr2=0x7f66d00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66d00a2c30 0x7f66d00a50f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66e8198ea0 tx=0x7f66dc005f90 comp rx=0 tx=0).stop 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e8102260 msgr2=0x7f66e8197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e8102260 0x7f66e8197990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66d80314e0 tx=0x7f66d80045e0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 shutdown_connections 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66d00a2c30 0x7f66d00a50f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66e81074c0 0x7f66e819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66e8104b90 0x7f66e8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 --2- 172.21.15.110:0/1271058887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e8102260 0x7f66e8197990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 >> 172.21.15.110:0/1271058887 conn(0x7f66e80fbc10 msgr2=0x7f66e8102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 shutdown_connections 2024-04-25T00:04:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:19.134+0000 7f66edc53640 1 -- 172.21.15.110:0/1271058887 wait complete. 2024-04-25T00:04:19.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:18 smithi188 ceph-mon[33285]: pgmap v419: 97 pgs: 97 active+clean; 988 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 2.04k op/s 2024-04-25T00:04:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 75.0 s, 1518.2 tps, lat 65.798 ms stddev 18.208 2024-04-25T00:04:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:21 smithi188 ceph-mon[33285]: pgmap v420: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.63k op/s 2024-04-25T00:04:22.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:21 smithi110 ceph-mon[29187]: pgmap v420: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.63k op/s 2024-04-25T00:04:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:21 smithi173 ceph-mon[32402]: pgmap v420: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.63k op/s 2024-04-25T00:04:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:23 smithi188 ceph-mon[33285]: pgmap v421: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 1.77k op/s 2024-04-25T00:04:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:23 smithi110 ceph-mon[29187]: pgmap v421: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 1.77k op/s 2024-04-25T00:04:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:23 smithi173 ceph-mon[32402]: pgmap v421: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 1.77k op/s 2024-04-25T00:04:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:25 smithi188 ceph-mon[33285]: pgmap v422: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-04-25T00:04:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:25 smithi110 ceph-mon[29187]: pgmap v422: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-04-25T00:04:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:25 smithi173 ceph-mon[32402]: pgmap v422: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 1.75k op/s 2024-04-25T00:04:26.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 80.0 s, 1210.4 tps, lat 82.265 ms stddev 33.878 2024-04-25T00:04:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:27 smithi188 ceph-mon[33285]: pgmap v423: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.60k op/s 2024-04-25T00:04:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:27 smithi110 ceph-mon[29187]: pgmap v423: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.60k op/s 2024-04-25T00:04:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:27 smithi173 ceph-mon[32402]: pgmap v423: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.60k op/s 2024-04-25T00:04:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:29 smithi188 ceph-mon[33285]: pgmap v424: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.78k op/s 2024-04-25T00:04:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:29 smithi110 ceph-mon[29187]: pgmap v424: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.78k op/s 2024-04-25T00:04:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:29 smithi173 ceph-mon[32402]: pgmap v424: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.78k op/s 2024-04-25T00:04:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:30 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:04:31.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:30 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:04:31.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:30 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:04:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 85.0 s, 810.2 tps, lat 123.284 ms stddev 15.059 2024-04-25T00:04:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:31 smithi188 ceph-mon[33285]: pgmap v425: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-04-25T00:04:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:31 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:04:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:31 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:04:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:31 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:04:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:31 smithi110 ceph-mon[29187]: pgmap v425: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-04-25T00:04:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:31 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:04:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:31 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:04:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:31 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:04:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:31 smithi173 ceph-mon[32402]: pgmap v425: 97 pgs: 97 active+clean; 988 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-04-25T00:04:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:31 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:04:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:31 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:04:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:31 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:04:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/3114863747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a381038b0 msgr2=0x7f7a38105ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/3114863747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a381038b0 0x7f7a38105ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009900 tx=0x7f7a2c02a410 comp rx=0 tx=0).stop 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/3114863747 shutdown_connections 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/3114863747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38106310 0x7f7a381087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/3114863747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a381038b0 0x7f7a38105ca0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/3114863747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a38100f80 0x7f7a38103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/3114863747 >> 172.21.15.110:0/3114863747 conn(0x7f7a380faf60 msgr2=0x7f7a380fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/3114863747 shutdown_connections 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.271+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/3114863747 wait complete. 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.272+0000 7f7a3cd9e640 1 Processor -- start 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.272+0000 7f7a3cd9e640 1 -- start start 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3cd9e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a38100f80 0x7f7a381979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3cd9e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a381038b0 0x7f7a38197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3cd9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38106310 0x7f7a3819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3cd9e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c220 con 0x7f7a381038b0 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3cd9e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c0a0 con 0x7f7a38100f80 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3cd9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c3a0 con 0x7f7a38106310 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a38100f80 0x7f7a381979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a38100f80 0x7f7a381979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:52156/0 (socket says 172.21.15.110:52156) 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a35d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a381038b0 0x7f7a38197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 -- 172.21.15.110:0/2363780104 learned_addr learned my addr 172.21.15.110:0/2363780104 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a36d9d640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38106310 0x7f7a3819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 -- 172.21.15.110:0/2363780104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38106310 msgr2=0x7f7a3819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38106310 0x7f7a3819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 -- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a381038b0 msgr2=0x7f7a38197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a381038b0 0x7f7a38197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 -- 172.21.15.110:0/2363780104 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3819f5c0 con 0x7f7a38100f80 2024-04-25T00:04:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.273+0000 7f7a3659c640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a38100f80 0x7f7a381979a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800e9f0 tx=0x7f7a2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.274+0000 7f7a277fe640 1 -- 172.21.15.110:0/2363780104 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800cd80 con 0x7f7a38100f80 2024-04-25T00:04:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.274+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/2363780104 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3819f8b0 con 0x7f7a38100f80 2024-04-25T00:04:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.274+0000 7f7a277fe640 1 -- 172.21.15.110:0/2363780104 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a280108e0 con 0x7f7a38100f80 2024-04-25T00:04:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.274+0000 7f7a277fe640 1 -- 172.21.15.110:0/2363780104 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28018920 con 0x7f7a38100f80 2024-04-25T00:04:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.274+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/2363780104 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a381015f0 con 0x7f7a38100f80 2024-04-25T00:04:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.276+0000 7f7a277fe640 1 -- 172.21.15.110:0/2363780104 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7a28010430 con 0x7f7a38100f80 2024-04-25T00:04:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.277+0000 7f7a277fe640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a200a2d80 0x7f7a200a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.277+0000 7f7a277fe640 1 -- 172.21.15.110:0/2363780104 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7a280cf2f0 con 0x7f7a38100f80 2024-04-25T00:04:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.277+0000 7f7a35d9b640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a200a2d80 0x7f7a200a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.277+0000 7f7a3fb84740 1 Processor -- start 2024-04-25T00:04:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3fb84740 1 -- start start 2024-04-25T00:04:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4a660 0x556396e4aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e4fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e52650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556396da5cb0 con 0x556396e4b050 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556396d4e5b0 con 0x556396e4a660 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556396da1c70 con 0x556396e501c0 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a36d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e52650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4a660 0x556396e4aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4a660 0x556396e4aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:52172/0 (socket says 172.21.15.110:52172) 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.278+0000 7f7a3659c640 1 -- 172.21.15.110:0/1103742844 learned_addr learned my addr 172.21.15.110:0/1103742844 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a3659c640 1 -- 172.21.15.110:0/1103742844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 msgr2=0x556396e52650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a3659c640 1 --2- 172.21.15.110:0/1103742844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e52650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a3659c640 1 -- 172.21.15.110:0/1103742844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 msgr2=0x556396e4fb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a3659c640 1 --2- 172.21.15.110:0/1103742844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e4fb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a3659c640 1 -- 172.21.15.110:0/1103742844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556396aba210 con 0x556396e4a660 2024-04-25T00:04:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a3659c640 1 --2- 172.21.15.110:0/1103742844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4a660 0x556396e4aa80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a280985a0 tx=0x7f7a2809b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6139d51eb305f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a35d9b640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a200a2d80 0x7f7a200a5240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009850 tx=0x7f7a2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a24ff9640 1 -- 172.21.15.110:0/1103742844 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2809fed0 con 0x556396e4a660 2024-04-25T00:04:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a24ff9640 1 -- 172.21.15.110:0/1103742844 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a2809ce50 con 0x556396e4a660 2024-04-25T00:04:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.279+0000 7f7a24ff9640 1 -- 172.21.15.110:0/1103742844 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28097920 con 0x556396e4a660 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 -- 172.21.15.110:0/1103742844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4a660 msgr2=0x556396e4aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/1103742844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4a660 0x556396e4aa80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a280985a0 tx=0x7f7a2809b840 comp rx=0 tx=0).stop 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 -- 172.21.15.110:0/1103742844 shutdown_connections 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/1103742844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e52650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/1103742844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e4fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/1103742844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4a660 0x556396e4aa80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 -- 172.21.15.110:0/1103742844 >> 172.21.15.110:0/1103742844 conn(0x556396e45cd0 msgr2=0x556396e460d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 -- 172.21.15.110:0/1103742844 shutdown_connections 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.280+0000 7f7a3fb84740 1 -- 172.21.15.110:0/1103742844 wait complete. 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 1 -- start start 2024-04-25T00:04:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 1 Processor -- start 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e55030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e5c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556396e53110 con 0x556396e4b050 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556396d61b50 con 0x556396e501c0 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556396e53290 con 0x556396e4a660 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3fb84740 10 client.0 _ll_register_callbacks cb 0x556396d9cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a35d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e55030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:55904/0 (socket says 172.21.15.110:55904) 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a3659c640 1 -- 172.21.15.110:0/2135170692 learned_addr learned my addr 172.21.15.110:0/2135170692 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.281+0000 7f7a36d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e5c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a3659c640 1 -- 172.21.15.110:0/2135170692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 msgr2=0x556396e5c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a3659c640 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e5c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a3659c640 1 -- 172.21.15.110:0/2135170692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 msgr2=0x556396e55030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a3659c640 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e55030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a3659c640 1 -- 172.21.15.110:0/2135170692 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556396b86620 con 0x556396e4a660 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a35d9b640 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e55030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a3659c640 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a28098030 tx=0x7f7a280cd0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b723f9da5943d1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a1dffb640 1 -- 172.21.15.110:0/2135170692 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2809fed0 con 0x556396e4a660 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a1dffb640 1 -- 172.21.15.110:0/2135170692 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7a280ba310 con 0x556396e4a660 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2135170692 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a2809ce50 con 0x556396e4a660 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.282+0000 7f7a1dffb640 1 -- 172.21.15.110:0/2135170692 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a280ba510 con 0x556396e4a660 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a3fb84740 10 client.25021 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2135170692 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556396e53110 con 0x556396e4a660 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a1dffb640 1 -- 172.21.15.110:0/2135170692 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a2809ce50 con 0x556396e4a660 2024-04-25T00:04:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a1dffb640 1 -- 172.21.15.110:0/2135170692 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f7a2809ac20 con 0x556396e4a660 2024-04-25T00:04:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a3fb84740 10 client.25021 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a3fb84740 10 client.25021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a3fb84740 20 client.25021 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556396e606c0 0x556396e62ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a3fb84740 4 client.25021 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2135170692 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556396db31f0 con 0x556396e606c0 2024-04-25T00:04:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.283+0000 7f7a35d9b640 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556396e606c0 0x556396e62ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.284+0000 7f7a1dffb640 10 client.25021 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.284+0000 7f7a35d9b640 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556396e606c0 0x556396e62ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445cfa9b571f0e86 server_cookie=9c87725d0eaf53a2 in_seq=0 out_seq=0 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.285+0000 7f7a1dffb640 1 -- 172.21.15.110:0/2135170692 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556396db31f0 con 0x556396e606c0 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.285+0000 7f7a1dffb640 10 client.25021 handle_command_reply: tid=0 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.285+0000 7f7a3fb84740 1 client.25021 shutdown 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.285+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2135170692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 msgr2=0x556396e54af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.285+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a28098030 tx=0x7f7a280cd0e0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2135170692 shutdown_connections 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556396e606c0 0x556396e62ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e5c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e55030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2135170692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2135170692 >> 172.21.15.110:0/2135170692 conn(0x556396e45cd0 msgr2=0x556396e3c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a1dffb640 0 client.25021 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2135170692 shutdown_connections 2024-04-25T00:04:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2135170692 wait complete. 2024-04-25T00:04:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.286+0000 7f7a3fb84740 20 client.25021 trim_cache size 0 max 16384 2024-04-25T00:04:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.309+0000 7f7a3fb84740 1 Processor -- start 2024-04-25T00:04:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.309+0000 7f7a3fb84740 1 -- start start 2024-04-25T00:04:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.309+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.309+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e3c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a3659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:55910/0 (socket says 172.21.15.110:55910) 2024-04-25T00:04:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a3659c640 1 -- 172.21.15.110:0/2965277203 learned_addr learned my addr 172.21.15.110:0/2965277203 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2965277203 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a9300 con 0x556396e4b050 2024-04-25T00:04:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2965277203 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a6b40 con 0x556396e501c0 2024-04-25T00:04:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2965277203 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a4990 con 0x556396e4a660 2024-04-25T00:04:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a36d9d640 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a35d9b640 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e3c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a36d9d640 1 -- 172.21.15.110:0/2965277203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 msgr2=0x556396e54f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a36d9d640 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a36d9d640 1 -- 172.21.15.110:0/2965277203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 msgr2=0x556396e3c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a36d9d640 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e3c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.310+0000 7f7a36d9d640 1 -- 172.21.15.110:0/2965277203 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556396e46a40 con 0x556396e501c0 2024-04-25T00:04:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.311+0000 7f7a36d9d640 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e673a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a30011540 tx=0x7f7a30009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632cb7cc8cfcf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.311+0000 7f7a1ffff640 1 -- 172.21.15.110:0/2965277203 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30004b20 con 0x556396e501c0 2024-04-25T00:04:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.311+0000 7f7a1ffff640 1 -- 172.21.15.110:0/2965277203 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a30004cc0 con 0x556396e501c0 2024-04-25T00:04:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.311+0000 7f7a1ffff640 1 -- 172.21.15.110:0/2965277203 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300053e0 con 0x556396e501c0 2024-04-25T00:04:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2965277203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 msgr2=0x556396e673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e673a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a30011540 tx=0x7f7a30009f90 comp rx=0 tx=0).stop 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2965277203 shutdown_connections 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e501c0 0x556396e673a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4b050 0x556396e3c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/2965277203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e4a660 0x556396e54f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2965277203 >> 172.21.15.110:0/2965277203 conn(0x556396e45cd0 msgr2=0x556396e3bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2965277203 shutdown_connections 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.312+0000 7f7a3fb84740 1 -- 172.21.15.110:0/2965277203 wait complete. 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.313+0000 7f7a3fb84740 1 -- start start 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.313+0000 7f7a3fb84740 1 Processor -- start 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.313+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4a660 0x556396e59280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.313+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4b050 0x556396e597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.313+0000 7f7a3fb84740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e5a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a5000 con 0x556396e4a660 2024-04-25T00:04:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a6470 con 0x556396e4b050 2024-04-25T00:04:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a3fb84740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a56d0 con 0x556396e501c0 2024-04-25T00:04:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a3fb84740 10 client.0 _ll_register_callbacks cb 0x7f7a2809b530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e5a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e5a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:55918/0 (socket says 172.21.15.110:55918) 2024-04-25T00:04:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 -- 172.21.15.110:0/3992582925 learned_addr learned my addr 172.21.15.110:0/3992582925 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a35d9b640 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4b050 0x556396e597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a3659c640 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4a660 0x556396e59280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 -- 172.21.15.110:0/3992582925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4b050 msgr2=0x556396e597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4b050 0x556396e597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 -- 172.21.15.110:0/3992582925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4a660 msgr2=0x556396e59280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4a660 0x556396e59280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 -- 172.21.15.110:0/3992582925 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556396e5adc0 con 0x556396e501c0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a36d9d640 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e5a7d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a3001f000 tx=0x7f7a30005f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b40dd2071e9e747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.314+0000 7f7a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a1dffb640 1 -- 172.21.15.110:0/3992582925 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3002a070 con 0x556396e501c0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a3fb84740 1 -- 172.21.15.110:0/3992582925 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a30004cc0 con 0x556396e501c0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a1dffb640 1 -- 172.21.15.110:0/3992582925 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7a3000f750 con 0x556396e501c0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a1dffb640 1 -- 172.21.15.110:0/3992582925 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a30036050 con 0x556396e501c0 2024-04-25T00:04:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a3fb84740 10 client.25027 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a3fb84740 1 -- 172.21.15.110:0/3992582925 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556396ec7240 con 0x556396e501c0 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a1dffb640 1 -- 172.21.15.110:0/3992582925 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a30004cc0 con 0x556396e501c0 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a1dffb640 1 -- 172.21.15.110:0/3992582925 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f7a300057e0 con 0x556396e501c0 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a3fb84740 10 client.25027 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a3fb84740 10 client.25027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.315+0000 7f7a3fb84740 20 client.25027 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.316+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556396ec7bb0 0x556396ec9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.316+0000 7f7a3659c640 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556396ec7bb0 0x556396ec9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.316+0000 7f7a3fb84740 4 client.25027 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.316+0000 7f7a3fb84740 1 -- 172.21.15.110:0/3992582925 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556396dad540 con 0x556396ec7bb0 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.316+0000 7f7a1dffb640 10 client.25027 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.316+0000 7f7a3659c640 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556396ec7bb0 0x556396ec9fd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10335c33770753a8 server_cookie=494653052ad0e034 in_seq=0 out_seq=0 2024-04-25T00:04:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.332+0000 7f7a1dffb640 1 -- 172.21.15.110:0/3992582925 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556396dad540 con 0x556396ec7bb0 2024-04-25T00:04:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a1dffb640 10 client.25027 handle_command_reply: tid=0 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 client.25027 shutdown 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 -- 172.21.15.110:0/3992582925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 msgr2=0x556396e5a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e5a7d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a3001f000 tx=0x7f7a30005f50 comp rx=0 tx=0).stop 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 -- 172.21.15.110:0/3992582925 shutdown_connections 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556396ec7bb0 0x556396ec9fd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556396e501c0 0x556396e5a7d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a1dffb640 0 client.25027 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556396e4b050 0x556396e597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 --2- 172.21.15.110:0/3992582925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556396e4a660 0x556396e59280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 -- 172.21.15.110:0/3992582925 >> 172.21.15.110:0/3992582925 conn(0x556396e45cd0 msgr2=0x556396e3c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 -- 172.21.15.110:0/3992582925 shutdown_connections 2024-04-25T00:04:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 1 -- 172.21.15.110:0/3992582925 wait complete. 2024-04-25T00:04:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.333+0000 7f7a3fb84740 20 client.25027 trim_cache size 0 max 16384 2024-04-25T00:04:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:04:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:04:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:04:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:04:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.334+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a200a2d80 msgr2=0x7f7a200a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.334+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a200a2d80 0x7f7a200a5240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009850 tx=0x7f7a2c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.334+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/2363780104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a38100f80 msgr2=0x7f7a381979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.334+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a38100f80 0x7f7a381979a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800e9f0 tx=0x7f7a2800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.334+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/2363780104 shutdown_connections 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.334+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7a200a2d80 0x7f7a200a5240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.334+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a38106310 0x7f7a3819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.335+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a381038b0 0x7f7a38197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.335+0000 7f7a3cd9e640 1 --2- 172.21.15.110:0/2363780104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a38100f80 0x7f7a381979a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.335+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/2363780104 >> 172.21.15.110:0/2363780104 conn(0x7f7a380faf60 msgr2=0x7f7a381001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.335+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/2363780104 shutdown_connections 2024-04-25T00:04:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:32.335+0000 7f7a3cd9e640 1 -- 172.21.15.110:0/2363780104 wait complete. 2024-04-25T00:04:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:33 smithi188 ceph-mon[33285]: pgmap v426: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.43k op/s 2024-04-25T00:04:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:33 smithi110 ceph-mon[29187]: pgmap v426: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.43k op/s 2024-04-25T00:04:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:33 smithi173 ceph-mon[32402]: pgmap v426: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.43k op/s 2024-04-25T00:04:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:35 smithi188 ceph-mon[33285]: pgmap v427: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-04-25T00:04:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:35 smithi110 ceph-mon[29187]: pgmap v427: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-04-25T00:04:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:35 smithi173 ceph-mon[32402]: pgmap v427: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 1.40k op/s 2024-04-25T00:04:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 90.0 s, 876.2 tps, lat 114.167 ms stddev 15.793 2024-04-25T00:04:37.904 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:37 smithi173 ceph-mon[32402]: pgmap v428: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-04-25T00:04:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:37 smithi188 ceph-mon[33285]: pgmap v428: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-04-25T00:04:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:37 smithi110 ceph-mon[29187]: pgmap v428: 97 pgs: 97 active+clean; 988 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-04-25T00:04:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:39 smithi188 ceph-mon[33285]: pgmap v429: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-04-25T00:04:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:39 smithi110 ceph-mon[29187]: pgmap v429: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-04-25T00:04:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:39 smithi173 ceph-mon[32402]: pgmap v429: 97 pgs: 97 active+clean; 988 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-04-25T00:04:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 95.0 s, 906.2 tps, lat 110.242 ms stddev 9.730 2024-04-25T00:04:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:41 smithi188 ceph-mon[33285]: pgmap v430: 97 pgs: 97 active+clean; 989 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 1.14k op/s 2024-04-25T00:04:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:41 smithi110 ceph-mon[29187]: pgmap v430: 97 pgs: 97 active+clean; 989 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 1.14k op/s 2024-04-25T00:04:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:41 smithi173 ceph-mon[32402]: pgmap v430: 97 pgs: 97 active+clean; 989 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 1.14k op/s 2024-04-25T00:04:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:43 smithi188 ceph-mon[33285]: pgmap v431: 97 pgs: 97 active+clean; 989 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-04-25T00:04:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:43 smithi110 ceph-mon[29187]: pgmap v431: 97 pgs: 97 active+clean; 989 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-04-25T00:04:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:43 smithi173 ceph-mon[32402]: pgmap v431: 97 pgs: 97 active+clean; 989 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 -- 172.21.15.110:0/3448766206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4102ae0 msgr2=0x7f36b4102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 --2- 172.21.15.110:0/3448766206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4102ae0 0x7f36b4102f60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009860 tx=0x7f36a402a320 comp rx=0 tx=0).stop 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 -- 172.21.15.110:0/3448766206 shutdown_connections 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 --2- 172.21.15.110:0/3448766206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b41034a0 0x7f36b4109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 --2- 172.21.15.110:0/3448766206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b4102ae0 0x7f36b4102f60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 --2- 172.21.15.110:0/3448766206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36b40ffa70 0x7f36b40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 -- 172.21.15.110:0/3448766206 >> 172.21.15.110:0/3448766206 conn(0x7f36b40fb3a0 msgr2=0x7f36b40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 -- 172.21.15.110:0/3448766206 shutdown_connections 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.448+0000 7f36bba72640 1 -- 172.21.15.110:0/3448766206 wait complete. 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36bba72640 1 Processor -- start 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36bba72640 1 -- start start 2024-04-25T00:04:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36bba72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36b40ffa70 0x7f36b4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36bba72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b4102ae0 0x7f36b4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36bba72640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b41034a0 0x7f36b419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36bba72640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36b410d2e0 con 0x7f36b41034a0 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36bba72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f36b410d160 con 0x7f36b40ffa70 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36bba72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36b410d460 con 0x7f36b4102ae0 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36b980e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36b40ffa70 0x7f36b4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36b900d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b4102ae0 0x7f36b4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36b980e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36b40ffa70 0x7f36b4197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:44230/0 (socket says 172.21.15.110:44230) 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36b900d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b4102ae0 0x7f36b4197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:42534/0 (socket says 172.21.15.110:42534) 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36b980e640 1 -- 172.21.15.110:0/1463505448 learned_addr learned my addr 172.21.15.110:0/1463505448 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.449+0000 7f36ba00f640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b41034a0 0x7f36b419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36ba00f640 1 -- 172.21.15.110:0/1463505448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b4102ae0 msgr2=0x7f36b4197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36ba00f640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b4102ae0 0x7f36b4197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36ba00f640 1 -- 172.21.15.110:0/1463505448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36b40ffa70 msgr2=0x7f36b4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36ba00f640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36b40ffa70 0x7f36b4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36ba00f640 1 -- 172.21.15.110:0/1463505448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b419f550 con 0x7f36b41034a0 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36b980e640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36b40ffa70 0x7f36b4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36ba00f640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b41034a0 0x7f36b419eee0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009fd0 tx=0x7f36ac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36aaffd640 1 -- 172.21.15.110:0/1463505448 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00ce20 con 0x7f36b41034a0 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36aaffd640 1 -- 172.21.15.110:0/1463505448 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36ac005020 con 0x7f36b41034a0 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36bba72640 1 -- 172.21.15.110:0/1463505448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b419f840 con 0x7f36b41034a0 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36aaffd640 1 -- 172.21.15.110:0/1463505448 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac010690 con 0x7f36b41034a0 2024-04-25T00:04:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.450+0000 7f36bba72640 1 -- 172.21.15.110:0/1463505448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b419fd10 con 0x7f36b41034a0 2024-04-25T00:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.453+0000 7f36aaffd640 1 -- 172.21.15.110:0/1463505448 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f36ac0051d0 con 0x7f36b41034a0 2024-04-25T00:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.453+0000 7f36be77c740 1 Processor -- start 2024-04-25T00:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.453+0000 7f36be77c740 1 -- start start 2024-04-25T00:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.453+0000 7f36be77c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e1e50 0x55b86f8e2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.453+0000 7f36be77c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e2840 0x55b86f8e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36be77c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8e79b0 0x55b86f8e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36be77c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b86f845cb0 con 0x55b86f8e2840 2024-04-25T00:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36be77c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b86f7ee5b0 con 0x55b86f8e79b0 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36be77c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b86f841c70 con 0x55b86f8e1e50 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e2840 0x55b86f8e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b900d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e1e50 0x55b86f8e2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e2840 0x55b86f8e7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56774/0 (socket says 172.21.15.110:56774) 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b900d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e1e50 0x55b86f8e2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:42544/0 (socket says 172.21.15.110:42544) 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 -- 172.21.15.110:0/1539609150 learned_addr learned my addr 172.21.15.110:0/1539609150 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36aaffd640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f36a00a2c30 0x7f36a00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b900d640 1 --2- 172.21.15.110:0/1539609150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8e79b0 0x55b86f8e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 -- 172.21.15.110:0/1539609150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e1e50 msgr2=0x55b86f8e2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 --2- 172.21.15.110:0/1539609150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e1e50 0x55b86f8e2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 -- 172.21.15.110:0/1539609150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8e79b0 msgr2=0x55b86f8e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 --2- 172.21.15.110:0/1539609150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8e79b0 0x55b86f8e9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36aaffd640 1 -- 172.21.15.110:0/1463505448 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f36ac0cf6e0 con 0x7f36b41034a0 2024-04-25T00:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36ba00f640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f36a00a2c30 0x7f36a00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 -- 172.21.15.110:0/1539609150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b86f55a210 con 0x55b86f8e2840 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.454+0000 7f36b980e640 1 --2- 172.21.15.110:0/1539609150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e2840 0x55b86f8e7340 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f36b00115c0 tx=0x7f36b0011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d85e6490db0f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f36ba00f640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f36a00a2c30 0x7f36a00a50f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f36ac01a040 tx=0x7f36ac005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f369ffff640 1 -- 172.21.15.110:0/1539609150 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0004020 con 0x55b86f8e2840 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f369ffff640 1 -- 172.21.15.110:0/1539609150 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b00041c0 con 0x55b86f8e2840 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f369ffff640 1 -- 172.21.15.110:0/1539609150 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0003920 con 0x55b86f8e2840 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f36be77c740 1 -- 172.21.15.110:0/1539609150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e2840 msgr2=0x55b86f8e7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f36be77c740 1 --2- 172.21.15.110:0/1539609150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e2840 0x55b86f8e7340 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f36b00115c0 tx=0x7f36b0011a80 comp rx=0 tx=0).stop 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f36be77c740 1 -- 172.21.15.110:0/1539609150 shutdown_connections 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f36be77c740 1 --2- 172.21.15.110:0/1539609150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8e79b0 0x55b86f8e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f36be77c740 1 --2- 172.21.15.110:0/1539609150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e2840 0x55b86f8e7340 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.455+0000 7f36be77c740 1 --2- 172.21.15.110:0/1539609150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e1e50 0x55b86f8e2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.456+0000 7f36be77c740 1 -- 172.21.15.110:0/1539609150 >> 172.21.15.110:0/1539609150 conn(0x55b86f8dd4c0 msgr2=0x55b86f8dd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.456+0000 7f36be77c740 1 -- 172.21.15.110:0/1539609150 shutdown_connections 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.456+0000 7f36be77c740 1 -- 172.21.15.110:0/1539609150 wait complete. 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.456+0000 7f36be77c740 1 -- start start 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.456+0000 7f36be77c740 1 Processor -- start 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.456+0000 7f36be77c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36be77c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36be77c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f8fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36be77c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b86f8ea960 con 0x55b86f8e1e50 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36be77c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b86f801b50 con 0x55b86f8fa210 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36be77c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b86f8eaae0 con 0x55b86f8e79b0 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36be77c740 10 client.0 _ll_register_callbacks cb 0x55b86f83cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b980e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b980e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:42550/0 (socket says 172.21.15.110:42550) 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b900d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b980e640 1 -- 172.21.15.110:0/4169761147 learned_addr learned my addr 172.21.15.110:0/4169761147 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b900d640 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f8fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b900d640 1 -- 172.21.15.110:0/4169761147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 msgr2=0x55b86f8f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b900d640 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b900d640 1 -- 172.21.15.110:0/4169761147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 msgr2=0x55b86f8fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b900d640 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f8fc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b900d640 1 -- 172.21.15.110:0/4169761147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b86f626620 con 0x55b86f8e1e50 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b980e640 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f369d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.457+0000 7f36b900d640 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f4ca0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009830 tx=0x7f36a4009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977eee513ea6ff00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f369d7fa640 1 -- 172.21.15.110:0/4169761147 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4037040 con 0x55b86f8e1e50 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 1 -- 172.21.15.110:0/4169761147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b00041c0 con 0x55b86f8e1e50 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f369d7fa640 1 -- 172.21.15.110:0/4169761147 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f36a4004240 con 0x55b86f8e1e50 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f369d7fa640 1 -- 172.21.15.110:0/4169761147 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36a4004440 con 0x55b86f8e1e50 2024-04-25T00:04:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 10 client.15477 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 1 -- 172.21.15.110:0/4169761147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b86f8ffa30 con 0x55b86f8e1e50 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f369d7fa640 1 -- 172.21.15.110:0/4169761147 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b00041c0 con 0x55b86f8e1e50 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f369d7fa640 1 -- 172.21.15.110:0/4169761147 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f36a403d040 con 0x55b86f8e1e50 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 20 client.15477 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b86f901510 0x55b86f9039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 4 client.15477 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36be77c740 1 -- 172.21.15.110:0/4169761147 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b86f8531f0 con 0x55b86f901510 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.458+0000 7f36b980e640 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b86f901510 0x55b86f9039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.459+0000 7f369d7fa640 10 client.15477 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.459+0000 7f36b980e640 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b86f901510 0x55b86f9039b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60d07a02aaed25c server_cookie=dc3f8b1c2e90794 in_seq=0 out_seq=0 2024-04-25T00:04:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f369d7fa640 1 -- 172.21.15.110:0/4169761147 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b86f8531f0 con 0x55b86f901510 2024-04-25T00:04:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f369d7fa640 10 client.15477 handle_command_reply: tid=0 2024-04-25T00:04:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 client.15477 shutdown 2024-04-25T00:04:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 -- 172.21.15.110:0/4169761147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 msgr2=0x55b86f8f4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f4ca0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009830 tx=0x7f36a4009590 comp rx=0 tx=0).stop 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 -- 172.21.15.110:0/4169761147 shutdown_connections 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b86f901510 0x55b86f9039b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f8fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f369d7fa640 0 client.15477 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 --2- 172.21.15.110:0/4169761147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f4ca0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 -- 172.21.15.110:0/4169761147 >> 172.21.15.110:0/4169761147 conn(0x55b86f8dd4c0 msgr2=0x55b86f8dc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 -- 172.21.15.110:0/4169761147 shutdown_connections 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 1 -- 172.21.15.110:0/4169761147 wait complete. 2024-04-25T00:04:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.460+0000 7f36be77c740 20 client.15477 trim_cache size 0 max 16384 2024-04-25T00:04:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.479+0000 7f36be77c740 1 Processor -- start 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.479+0000 7f36be77c740 1 -- start start 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36be77c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36be77c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36be77c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f8dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36be77c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41a9110 con 0x55b86f8e1e50 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36be77c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41a6950 con 0x55b86f8fa210 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36be77c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41a47a0 con 0x55b86f8e79b0 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b980e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56792/0 (socket says 172.21.15.110:56792) 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 -- 172.21.15.110:0/3577770160 learned_addr learned my addr 172.21.15.110:0/3577770160 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 --2- 172.21.15.110:0/3577770160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f8dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 -- 172.21.15.110:0/3577770160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 msgr2=0x55b86f8dd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 --2- 172.21.15.110:0/3577770160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8dd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 -- 172.21.15.110:0/3577770160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 msgr2=0x55b86f8dbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 --2- 172.21.15.110:0/3577770160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f8dbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 -- 172.21.15.110:0/3577770160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b86f8e76f0 con 0x55b86f8e1e50 2024-04-25T00:04:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f36b900d640 1 --2- 172.21.15.110:0/3577770160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f5140 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f36a40059f0 tx=0x7f36a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b475020409dd678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.480+0000 7f369f7fe640 1 -- 172.21.15.110:0/3577770160 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4037070 con 0x55b86f8e1e50 2024-04-25T00:04:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f369f7fe640 1 -- 172.21.15.110:0/3577770160 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36a404fa10 con 0x55b86f8e1e50 2024-04-25T00:04:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f369f7fe640 1 -- 172.21.15.110:0/3577770160 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a403f6d0 con 0x55b86f8e1e50 2024-04-25T00:04:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 -- 172.21.15.110:0/3577770160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 msgr2=0x55b86f8f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 --2- 172.21.15.110:0/3577770160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f5140 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f36a40059f0 tx=0x7f36a4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 -- 172.21.15.110:0/3577770160 shutdown_connections 2024-04-25T00:04:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 --2- 172.21.15.110:0/3577770160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f8dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 --2- 172.21.15.110:0/3577770160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f8dd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 --2- 172.21.15.110:0/3577770160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f8f5140 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 -- 172.21.15.110:0/3577770160 >> 172.21.15.110:0/3577770160 conn(0x55b86f8dd4c0 msgr2=0x55b86f8f3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 -- 172.21.15.110:0/3577770160 shutdown_connections 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.481+0000 7f36be77c740 1 -- 172.21.15.110:0/3577770160 wait complete. 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 1 -- start start 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 1 Processor -- start 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f908120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f908660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f965950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41a4e10 con 0x55b86f8e1e50 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41a6280 con 0x55b86f8fa210 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41a54e0 con 0x55b86f8e79b0 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36be77c740 10 client.0 _ll_register_callbacks cb 0x7f36a40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.482+0000 7f36b900d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f908120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b980e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f908660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b900d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f908120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56800/0 (socket says 172.21.15.110:56800) 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b900d640 1 -- 172.21.15.110:0/2999799355 learned_addr learned my addr 172.21.15.110:0/2999799355 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b900d640 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f965950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b980e640 1 -- 172.21.15.110:0/2999799355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 msgr2=0x55b86f965950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b980e640 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f965950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b980e640 1 -- 172.21.15.110:0/2999799355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 msgr2=0x55b86f908120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b980e640 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f908120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b980e640 1 -- 172.21.15.110:0/2999799355 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b86f908f10 con 0x55b86f8e79b0 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b900d640 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f908120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36b980e640 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f908660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b86f8f6180 tx=0x7f36b0009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=112eb5a1de72ba1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f369d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f369d7fa640 1 -- 172.21.15.110:0/2999799355 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b001d040 con 0x55b86f8e79b0 2024-04-25T00:04:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f36be77c740 1 -- 172.21.15.110:0/2999799355 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b86f84d540 con 0x55b86f8e79b0 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.483+0000 7f369d7fa640 1 -- 172.21.15.110:0/2999799355 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f36b0009260 con 0x55b86f8e79b0 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f369d7fa640 1 -- 172.21.15.110:0/2999799355 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36b000f950 con 0x55b86f8e79b0 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36be77c740 10 client.25045 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36be77c740 1 -- 172.21.15.110:0/2999799355 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b86f966ff0 con 0x55b86f8e79b0 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f369d7fa640 1 -- 172.21.15.110:0/2999799355 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b86f84d540 con 0x55b86f8e79b0 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f369d7fa640 1 -- 172.21.15.110:0/2999799355 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f36b0032030 con 0x55b86f8e79b0 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36be77c740 10 client.25045 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36be77c740 10 client.25045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36be77c740 20 client.25045 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36be77c740 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b86f9678e0 0x55b86f969d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36be77c740 4 client.25045 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36be77c740 1 -- 172.21.15.110:0/2999799355 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b86f7cb8e0 con 0x55b86f9678e0 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.484+0000 7f36b900d640 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b86f9678e0 0x55b86f969d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.485+0000 7f369d7fa640 10 client.25045 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.485+0000 7f36b900d640 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b86f9678e0 0x55b86f969d00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce2c76145959a80 server_cookie=a7da2226a48a4bfe in_seq=0 out_seq=0 2024-04-25T00:04:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f369d7fa640 1 -- 172.21.15.110:0/2999799355 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b86f7cb8e0 con 0x55b86f9678e0 2024-04-25T00:04:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f369d7fa640 10 client.25045 handle_command_reply: tid=0 2024-04-25T00:04:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f36be77c740 1 client.25045 shutdown 2024-04-25T00:04:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f36be77c740 1 -- 172.21.15.110:0/2999799355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 msgr2=0x55b86f908660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f36be77c740 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f908660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b86f8f6180 tx=0x7f36b0009fd0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f36be77c740 1 -- 172.21.15.110:0/2999799355 shutdown_connections 2024-04-25T00:04:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f36be77c740 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b86f9678e0 0x55b86f969d00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f369d7fa640 0 client.25045 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f36be77c740 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b86f8fa210 0x55b86f965950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f36be77c740 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b86f8e79b0 0x55b86f908660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.505+0000 7f36be77c740 1 --2- 172.21.15.110:0/2999799355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b86f8e1e50 0x55b86f908120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.506+0000 7f36be77c740 1 -- 172.21.15.110:0/2999799355 >> 172.21.15.110:0/2999799355 conn(0x55b86f8dd4c0 msgr2=0x55b86f8dd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.506+0000 7f36be77c740 1 -- 172.21.15.110:0/2999799355 shutdown_connections 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.506+0000 7f36be77c740 1 -- 172.21.15.110:0/2999799355 wait complete. 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.506+0000 7f36be77c740 20 client.25045 trim_cache size 0 max 16384 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:04:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:04:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.506+0000 7f36bba72640 1 -- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f36a00a2c30 msgr2=0x7f36a00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.506+0000 7f36bba72640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f36a00a2c30 0x7f36a00a50f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f36ac01a040 tx=0x7f36ac005f70 comp rx=0 tx=0).stop 2024-04-25T00:04:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.506+0000 7f36bba72640 1 -- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b41034a0 msgr2=0x7f36b419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.506+0000 7f36bba72640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b41034a0 0x7f36b419eee0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009fd0 tx=0x7f36ac00ef10 comp rx=0 tx=0).stop 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.507+0000 7f36bba72640 1 -- 172.21.15.110:0/1463505448 shutdown_connections 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.507+0000 7f36bba72640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f36a00a2c30 0x7f36a00a50f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.507+0000 7f36bba72640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36b41034a0 0x7f36b419eee0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.507+0000 7f36bba72640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36b4102ae0 0x7f36b4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.507+0000 7f36bba72640 1 --2- 172.21.15.110:0/1463505448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36b40ffa70 0x7f36b4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.507+0000 7f36bba72640 1 -- 172.21.15.110:0/1463505448 >> 172.21.15.110:0/1463505448 conn(0x7f36b40fb3a0 msgr2=0x7f36b4105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.507+0000 7f36bba72640 1 -- 172.21.15.110:0/1463505448 shutdown_connections 2024-04-25T00:04:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:45.507+0000 7f36bba72640 1 -- 172.21.15.110:0/1463505448 wait complete. 2024-04-25T00:04:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:45 smithi188 ceph-mon[33285]: pgmap v432: 97 pgs: 97 active+clean; 989 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-04-25T00:04:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:45 smithi110 ceph-mon[29187]: pgmap v432: 97 pgs: 97 active+clean; 989 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-04-25T00:04:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:45 smithi173 ceph-mon[32402]: pgmap v432: 97 pgs: 97 active+clean; 989 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 10 MiB/s wr, 1.23k op/s 2024-04-25T00:04:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 100.0 s, 835.8 tps, lat 119.864 ms stddev 75.337 2024-04-25T00:04:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:47 smithi188 ceph-mon[33285]: pgmap v433: 97 pgs: 97 active+clean; 993 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.9 MiB/s wr, 1.03k op/s 2024-04-25T00:04:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:47 smithi110 ceph-mon[29187]: pgmap v433: 97 pgs: 97 active+clean; 993 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.9 MiB/s wr, 1.03k op/s 2024-04-25T00:04:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:47 smithi173 ceph-mon[32402]: pgmap v433: 97 pgs: 97 active+clean; 993 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.9 MiB/s wr, 1.03k op/s 2024-04-25T00:04:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:49 smithi188 ceph-mon[33285]: pgmap v434: 97 pgs: 97 active+clean; 1001 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-04-25T00:04:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:49 smithi110 ceph-mon[29187]: pgmap v434: 97 pgs: 97 active+clean; 1001 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-04-25T00:04:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:49 smithi173 ceph-mon[32402]: pgmap v434: 97 pgs: 97 active+clean; 1001 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-04-25T00:04:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 105.0 s, 839.8 tps, lat 119.017 ms stddev 8.711 2024-04-25T00:04:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:51 smithi188 ceph-mon[33285]: pgmap v435: 97 pgs: 97 active+clean; 1005 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:04:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:51 smithi110 ceph-mon[29187]: pgmap v435: 97 pgs: 97 active+clean; 1005 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:04:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:51 smithi173 ceph-mon[32402]: pgmap v435: 97 pgs: 97 active+clean; 1005 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:04:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:53 smithi188 ceph-mon[33285]: pgmap v436: 97 pgs: 97 active+clean; 1005 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-04-25T00:04:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:53 smithi110 ceph-mon[29187]: pgmap v436: 97 pgs: 97 active+clean; 1005 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-04-25T00:04:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:53 smithi173 ceph-mon[32402]: pgmap v436: 97 pgs: 97 active+clean; 1005 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-04-25T00:04:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:55 smithi188 ceph-mon[33285]: pgmap v437: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-04-25T00:04:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:55 smithi110 ceph-mon[29187]: pgmap v437: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-04-25T00:04:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:55 smithi173 ceph-mon[32402]: pgmap v437: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-04-25T00:04:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 110.0 s, 841.2 tps, lat 118.862 ms stddev 10.295 2024-04-25T00:04:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:57 smithi188 ceph-mon[33285]: pgmap v438: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.4 MiB/s wr, 979 op/s 2024-04-25T00:04:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:57 smithi110 ceph-mon[29187]: pgmap v438: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.4 MiB/s wr, 979 op/s 2024-04-25T00:04:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:57 smithi173 ceph-mon[32402]: pgmap v438: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.4 MiB/s wr, 979 op/s 2024-04-25T00:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 -- 172.21.15.110:0/1697204605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa478105570 msgr2=0x7fa47810be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 --2- 172.21.15.110:0/1697204605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa478105570 0x7fa47810be00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa470009be0 tx=0x7fa47002a6c0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 -- 172.21.15.110:0/1697204605 shutdown_connections 2024-04-25T00:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 --2- 172.21.15.110:0/1697204605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa478105570 0x7fa47810be00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 --2- 172.21.15.110:0/1697204605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa478104bb0 0x7fa478105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 --2- 172.21.15.110:0/1697204605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4781039c0 0x7fa478103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 -- 172.21.15.110:0/1697204605 >> 172.21.15.110:0/1697204605 conn(0x7fa4780fef30 msgr2=0x7fa478101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 -- 172.21.15.110:0/1697204605 shutdown_connections 2024-04-25T00:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.641+0000 7fa47d989640 1 -- 172.21.15.110:0/1697204605 wait complete. 2024-04-25T00:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa47d989640 1 Processor -- start 2024-04-25T00:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa47d989640 1 -- start start 2024-04-25T00:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa47d989640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4781039c0 0x7fa478199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa47d989640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa478104bb0 0x7fa478199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa47d989640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa478105570 0x7fa4781a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa47d989640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa47810f4d0 con 0x7fa478105570 2024-04-25T00:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa47d989640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa47810f350 con 0x7fa4781039c0 2024-04-25T00:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa47d989640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa47810f650 con 0x7fa478104bb0 2024-04-25T00:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.642+0000 7fa4777fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa478105570 0x7fa4781a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4781039c0 0x7fa478199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4777fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa478105570 0x7fa4781a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57364/0 (socket says 172.21.15.110:57364) 2024-04-25T00:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4777fe640 1 -- 172.21.15.110:0/1865352137 learned_addr learned my addr 172.21.15.110:0/1865352137 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4767fc640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa478104bb0 0x7fa478199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4777fe640 1 -- 172.21.15.110:0/1865352137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa478104bb0 msgr2=0x7fa478199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4777fe640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa478104bb0 0x7fa478199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4777fe640 1 -- 172.21.15.110:0/1865352137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4781039c0 msgr2=0x7fa478199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4777fe640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4781039c0 0x7fa478199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4777fe640 1 -- 172.21.15.110:0/1865352137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4781a16d0 con 0x7fa478105570 2024-04-25T00:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.643+0000 7fa4777fe640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa478105570 0x7fa4781a1060 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa470034040 tx=0x7fa470030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.644+0000 7fa47c987640 1 -- 172.21.15.110:0/1865352137 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470041020 con 0x7fa478105570 2024-04-25T00:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.644+0000 7fa47d989640 1 -- 172.21.15.110:0/1865352137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781a1960 con 0x7fa478105570 2024-04-25T00:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.644+0000 7fa47c987640 1 -- 172.21.15.110:0/1865352137 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa470037040 con 0x7fa478105570 2024-04-25T00:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.644+0000 7fa47d989640 1 -- 172.21.15.110:0/1865352137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781a1ea0 con 0x7fa478105570 2024-04-25T00:04:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.644+0000 7fa47c987640 1 -- 172.21.15.110:0/1865352137 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470036070 con 0x7fa478105570 2024-04-25T00:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.647+0000 7fa480772740 1 Processor -- start 2024-04-25T00:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.648+0000 7fa47c987640 1 -- 172.21.15.110:0/1865352137 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa470044020 con 0x7fa478105570 2024-04-25T00:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.648+0000 7fa480772740 1 -- start start 2024-04-25T00:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.648+0000 7fa480772740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf3660 0x5654bbdf3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.648+0000 7fa480772740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbdf4050 0x5654bbdf8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.648+0000 7fa480772740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 0x5654bbdfb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.648+0000 7fa480772740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654bbd4ecb0 con 0x5654bbdf3660 2024-04-25T00:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.648+0000 7fa480772740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654bbcf75b0 con 0x5654bbdf4050 2024-04-25T00:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.648+0000 7fa480772740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654bbd4ac70 con 0x5654bbdf91c0 2024-04-25T00:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbdf4050 0x5654bbdf8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa4767fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf3660 0x5654bbdf3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbdf4050 0x5654bbdf8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:55438/0 (socket says 172.21.15.110:55438) 2024-04-25T00:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa476ffd640 1 -- 172.21.15.110:0/2764263701 learned_addr learned my addr 172.21.15.110:0/2764263701 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa4767fc640 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 0x5654bbdfb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa4767fc640 1 -- 172.21.15.110:0/2764263701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 msgr2=0x5654bbdfb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa4767fc640 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 0x5654bbdfb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa4767fc640 1 -- 172.21.15.110:0/2764263701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbdf4050 msgr2=0x5654bbdf8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa4767fc640 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbdf4050 0x5654bbdf8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa4767fc640 1 -- 172.21.15.110:0/2764263701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bba63210 con 0x5654bbdf3660 2024-04-25T00:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa47c987640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa4600a2cb0 0x7fa4600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa476ffd640 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbdf4050 0x5654bbdf8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa4767fc640 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf3660 0x5654bbdf3a80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa47819afc0 tx=0x7fa46c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff7569baed1f412 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa476ffd640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa4600a2cb0 0x7fa4600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.649+0000 7fa47c987640 1 -- 172.21.15.110:0/1865352137 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa4700ef7b0 con 0x7fa478105570 2024-04-25T00:04:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.650+0000 7fa45dffb640 1 -- 172.21.15.110:0/2764263701 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c004020 con 0x5654bbdf3660 2024-04-25T00:04:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.650+0000 7fa45dffb640 1 -- 172.21.15.110:0/2764263701 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa46c0041c0 con 0x5654bbdf3660 2024-04-25T00:04:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.650+0000 7fa45dffb640 1 -- 172.21.15.110:0/2764263701 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00f880 con 0x5654bbdf3660 2024-04-25T00:04:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 -- 172.21.15.110:0/2764263701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf3660 msgr2=0x5654bbdf3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf3660 0x5654bbdf3a80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa47819afc0 tx=0x7fa46c005a00 comp rx=0 tx=0).stop 2024-04-25T00:04:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa476ffd640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa4600a2cb0 0x7fa4600a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5654bbdf4450 tx=0x7fa468005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 -- 172.21.15.110:0/2764263701 shutdown_connections 2024-04-25T00:04:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 0x5654bbdfb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbdf4050 0x5654bbdf8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 --2- 172.21.15.110:0/2764263701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf3660 0x5654bbdf3a80 secure :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa47819afc0 tx=0x7fa46c005a00 comp rx=0 tx=0).stop 2024-04-25T00:04:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 -- 172.21.15.110:0/2764263701 >> 172.21.15.110:0/2764263701 conn(0x5654bbdeecd0 msgr2=0x5654bbdef0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 -- 172.21.15.110:0/2764263701 shutdown_connections 2024-04-25T00:04:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 -- 172.21.15.110:0/2764263701 wait complete. 2024-04-25T00:04:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.651+0000 7fa480772740 1 -- start start 2024-04-25T00:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa480772740 1 Processor -- start 2024-04-25T00:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa480772740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf4050 0x5654bbdfdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa480772740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 0x5654bbdfe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf4050 0x5654bbdfdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf4050 0x5654bbdfdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57382/0 (socket says 172.21.15.110:57382) 2024-04-25T00:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa476ffd640 1 -- 172.21.15.110:0/3999058738 learned_addr learned my addr 172.21.15.110:0/3999058738 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa480772740 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa4767fc640 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 0x5654bbdfe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654bbdfc110 con 0x5654bbdf4050 2024-04-25T00:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654bbd0ab50 con 0x5654bbe032b0 2024-04-25T00:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.652+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654bbdfc290 con 0x5654bbdf91c0 2024-04-25T00:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.653+0000 7fa4777fe640 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.653+0000 7fa480772740 10 client.0 _ll_register_callbacks cb 0x5654bbd45ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.653+0000 7fa476ffd640 1 -- 172.21.15.110:0/3999058738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 msgr2=0x5654bbdfe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.653+0000 7fa476ffd640 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 0x5654bbdfe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.653+0000 7fa476ffd640 1 -- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 msgr2=0x5654bbe056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.653+0000 7fa476ffd640 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.653+0000 7fa476ffd640 1 -- 172.21.15.110:0/3999058738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bbb2f620 con 0x5654bbdf4050 2024-04-25T00:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.653+0000 7fa4777fe640 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.654+0000 7fa476ffd640 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf4050 0x5654bbdfdd40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa46800fa50 tx=0x7fa468004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff1633c818fab86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.654+0000 7fa456ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.654+0000 7fa456ffd640 1 -- 172.21.15.110:0/3999058738 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46802d060 con 0x5654bbdf4050 2024-04-25T00:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.654+0000 7fa456ffd640 1 -- 172.21.15.110:0/3999058738 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa468018030 con 0x5654bbdf4050 2024-04-25T00:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.654+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46c0041c0 con 0x5654bbdf4050 2024-04-25T00:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.654+0000 7fa456ffd640 1 -- 172.21.15.110:0/3999058738 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa46801d620 con 0x5654bbdf4050 2024-04-25T00:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.654+0000 7fa480772740 10 client.15495 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.654+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654bbe08aa0 con 0x5654bbdf4050 2024-04-25T00:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa456ffd640 1 -- 172.21.15.110:0/3999058738 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa46c0041c0 con 0x5654bbdf4050 2024-04-25T00:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa456ffd640 1 -- 172.21.15.110:0/3999058738 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fa468034850 con 0x5654bbdf4050 2024-04-25T00:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa480772740 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa480772740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa480772740 20 client.15495 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa480772740 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5654bbe0a250 0x5654bbe0c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa480772740 4 client.15495 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:04:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654bbd5c1f0 con 0x5654bbe0a250 2024-04-25T00:04:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.655+0000 7fa4767fc640 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5654bbe0a250 0x5654bbe0c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.656+0000 7fa456ffd640 10 client.15495 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.656+0000 7fa4767fc640 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5654bbe0a250 0x5654bbe0c710 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a03eb34c69672d server_cookie=58c14f5f137ff019 in_seq=0 out_seq=0 2024-04-25T00:04:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.656+0000 7fa456ffd640 1 -- 172.21.15.110:0/3999058738 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654bbd5c1f0 con 0x5654bbe0a250 2024-04-25T00:04:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.656+0000 7fa456ffd640 10 client.15495 handle_command_reply: tid=0 2024-04-25T00:04:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.656+0000 7fa480772740 1 client.15495 shutdown 2024-04-25T00:04:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf4050 msgr2=0x5654bbdfdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf4050 0x5654bbdfdd40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa46800fa50 tx=0x7fa468004730 comp rx=0 tx=0).stop 2024-04-25T00:04:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 shutdown_connections 2024-04-25T00:04:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5654bbe0a250 0x5654bbe0c710 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf91c0 0x5654bbdfe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 --2- 172.21.15.110:0/3999058738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf4050 0x5654bbdfdd40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 >> 172.21.15.110:0/3999058738 conn(0x5654bbdeecd0 msgr2=0x5654bbde55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa456ffd640 0 client.15495 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.657+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 shutdown_connections 2024-04-25T00:04:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.658+0000 7fa480772740 1 -- 172.21.15.110:0/3999058738 wait complete. 2024-04-25T00:04:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.658+0000 7fa480772740 20 client.15495 trim_cache size 0 max 16384 2024-04-25T00:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.683+0000 7fa480772740 1 Processor -- start 2024-04-25T00:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.683+0000 7fa480772740 1 -- start start 2024-04-25T00:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.683+0000 7fa480772740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe0c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.683+0000 7fa480772740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbdf5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa480772740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbdf5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa480772740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa464010cf0 con 0x5654bbdf91c0 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa480772740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa46400e530 con 0x5654bbe032b0 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa480772740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa46400c380 con 0x5654bbdf4050 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe0c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe0c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:53610/0 (socket says 172.21.15.110:53610) 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 -- 172.21.15.110:0/3123102640 learned_addr learned my addr 172.21.15.110:0/3123102640 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa4767fc640 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbdf5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa4777fe640 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbdf5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 -- 172.21.15.110:0/3123102640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 msgr2=0x5654bbdf5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbdf5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 -- 172.21.15.110:0/3123102640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 msgr2=0x5654bbdf5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbdf5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 -- 172.21.15.110:0/3123102640 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bbdf89e0 con 0x5654bbdf4050 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa4767fc640 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbdf5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa476ffd640 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe0c260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa468019040 tx=0x7fa46800ff10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac8c8ebc4dae3486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.684+0000 7fa45d7fa640 1 -- 172.21.15.110:0/3123102640 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468004250 con 0x5654bbdf4050 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.685+0000 7fa45d7fa640 1 -- 172.21.15.110:0/3123102640 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa468039020 con 0x5654bbdf4050 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.685+0000 7fa45d7fa640 1 -- 172.21.15.110:0/3123102640 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4680287c0 con 0x5654bbdf4050 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.685+0000 7fa480772740 1 -- 172.21.15.110:0/3123102640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 msgr2=0x5654bbe0c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.685+0000 7fa480772740 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe0c260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa468019040 tx=0x7fa46800ff10 comp rx=0 tx=0).stop 2024-04-25T00:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.685+0000 7fa480772740 1 -- 172.21.15.110:0/3123102640 shutdown_connections 2024-04-25T00:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.686+0000 7fa480772740 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbdf5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.686+0000 7fa480772740 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbdf5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.686+0000 7fa480772740 1 --2- 172.21.15.110:0/3123102640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe0c260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.686+0000 7fa480772740 1 -- 172.21.15.110:0/3123102640 >> 172.21.15.110:0/3123102640 conn(0x5654bbdeecd0 msgr2=0x5654bbdef0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.686+0000 7fa480772740 1 -- 172.21.15.110:0/3123102640 shutdown_connections 2024-04-25T00:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.686+0000 7fa480772740 1 -- 172.21.15.110:0/3123102640 wait complete. 2024-04-25T00:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.686+0000 7fa480772740 1 -- start start 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.687+0000 7fa480772740 1 Processor -- start 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.687+0000 7fa480772740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.687+0000 7fa480772740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbe1af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.687+0000 7fa480772740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa480772740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa46400c9f0 con 0x5654bbdf91c0 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa480772740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa46400de60 con 0x5654bbe032b0 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa480772740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa46400d0c0 con 0x5654bbdf4050 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe1aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:53616/0 (socket says 172.21.15.110:53616) 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 -- 172.21.15.110:0/4272001352 learned_addr learned my addr 172.21.15.110:0/4272001352 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa480772740 10 client.0 _ll_register_callbacks cb 0x7fa468009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa4767fc640 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbe1af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa4777fe640 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 -- 172.21.15.110:0/4272001352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 msgr2=0x5654bbe6e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe6e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 -- 172.21.15.110:0/4272001352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 msgr2=0x5654bbe1af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbe1af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 -- 172.21.15.110:0/4272001352 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bbe6e8a0 con 0x5654bbdf4050 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa476ffd640 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe1aa10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa468012d70 tx=0x7fa4680348b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5d93552b737011 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.688+0000 7fa456ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa456ffd640 1 -- 172.21.15.110:0/4272001352 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468005250 con 0x5654bbdf4050 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa456ffd640 1 -- 172.21.15.110:0/4272001352 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa4680053f0 con 0x5654bbdf4050 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa480772740 1 -- 172.21.15.110:0/4272001352 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa468039020 con 0x5654bbdf4050 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa456ffd640 1 -- 172.21.15.110:0/4272001352 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4680055f0 con 0x5654bbdf4050 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa480772740 10 client.25063 fetch_fsmap learned FSMap version 27 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa480772740 1 -- 172.21.15.110:0/4272001352 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654bbe6fa90 con 0x5654bbdf4050 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa456ffd640 1 -- 172.21.15.110:0/4272001352 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa468039020 con 0x5654bbdf4050 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa456ffd640 1 -- 172.21.15.110:0/4272001352 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fa46801d8c0 con 0x5654bbdf4050 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa480772740 10 client.25063 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa480772740 10 client.25063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa480772740 20 client.25063 populate_metadata read hostname 'smithi110' 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.689+0000 7fa480772740 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5654bbe70410 0x5654bbe72830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.690+0000 7fa480772740 4 client.25063 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.690+0000 7fa480772740 1 -- 172.21.15.110:0/4272001352 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654bbd56540 con 0x5654bbe70410 2024-04-25T00:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.690+0000 7fa4767fc640 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5654bbe70410 0x5654bbe72830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:04:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.691+0000 7fa4767fc640 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5654bbe70410 0x5654bbe72830 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a8f04c17ce17d3 server_cookie=7e98ccf84910336f in_seq=0 out_seq=0 2024-04-25T00:04:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.691+0000 7fa456ffd640 10 client.25063 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.709+0000 7fa456ffd640 1 -- 172.21.15.110:0/4272001352 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654bbd56540 con 0x5654bbe70410 2024-04-25T00:04:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.709+0000 7fa456ffd640 10 client.25063 handle_command_reply: tid=0 2024-04-25T00:04:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.709+0000 7fa480772740 1 client.25063 shutdown 2024-04-25T00:04:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.709+0000 7fa480772740 1 -- 172.21.15.110:0/4272001352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 msgr2=0x5654bbe1aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.709+0000 7fa480772740 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe1aa10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa468012d70 tx=0x7fa4680348b0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 1 -- 172.21.15.110:0/4272001352 shutdown_connections 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5654bbe70410 0x5654bbe72830 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654bbe032b0 0x5654bbe6e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bbdf91c0 0x5654bbe1af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 1 --2- 172.21.15.110:0/4272001352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654bbdf4050 0x5654bbe1aa10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 1 -- 172.21.15.110:0/4272001352 >> 172.21.15.110:0/4272001352 conn(0x5654bbdeecd0 msgr2=0x5654bbe0c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa456ffd640 0 client.25063 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 1 -- 172.21.15.110:0/4272001352 shutdown_connections 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 1 -- 172.21.15.110:0/4272001352 wait complete. 2024-04-25T00:04:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.710+0000 7fa480772740 20 client.25063 trim_cache size 0 max 16384 2024-04-25T00:04:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:04:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:04:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:04:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:04:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 -- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa4600a2cb0 msgr2=0x7fa4600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa4600a2cb0 0x7fa4600a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5654bbdf4450 tx=0x7fa468005f90 comp rx=0 tx=0).stop 2024-04-25T00:04:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 -- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa478105570 msgr2=0x7fa4781a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:04:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa478105570 0x7fa4781a1060 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa470034040 tx=0x7fa470030f90 comp rx=0 tx=0).stop 2024-04-25T00:04:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 -- 172.21.15.110:0/1865352137 shutdown_connections 2024-04-25T00:04:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa4600a2cb0 0x7fa4600a5170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa478105570 0x7fa4781a1060 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa478104bb0 0x7fa478199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 --2- 172.21.15.110:0/1865352137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4781039c0 0x7fa478199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:04:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 -- 172.21.15.110:0/1865352137 >> 172.21.15.110:0/1865352137 conn(0x7fa4780fef30 msgr2=0x7fa478107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:04:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 -- 172.21.15.110:0/1865352137 shutdown_connections 2024-04-25T00:04:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:04:58.711+0000 7fa47d989640 1 -- 172.21.15.110:0/1865352137 wait complete. 2024-04-25T00:05:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:04:59 smithi188 ceph-mon[33285]: pgmap v439: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-04-25T00:05:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:04:59 smithi110 ceph-mon[29187]: pgmap v439: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-04-25T00:05:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:04:59 smithi173 ceph-mon[32402]: pgmap v439: 97 pgs: 97 active+clean; 1006 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-04-25T00:05:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 115.0 s, 803.4 tps, lat 124.411 ms stddev 16.289 2024-04-25T00:05:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:01 smithi188 ceph-mon[33285]: pgmap v440: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 1.04k op/s 2024-04-25T00:05:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:01 smithi110 ceph-mon[29187]: pgmap v440: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 1.04k op/s 2024-04-25T00:05:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:01 smithi173 ceph-mon[32402]: pgmap v440: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 1.04k op/s 2024-04-25T00:05:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:03 smithi188 ceph-mon[33285]: pgmap v441: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:05:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:03 smithi110 ceph-mon[29187]: pgmap v441: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:05:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:03 smithi173 ceph-mon[32402]: pgmap v441: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:05:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:05 smithi188 ceph-mon[33285]: pgmap v442: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 945 op/s 2024-04-25T00:05:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:05 smithi110 ceph-mon[29187]: pgmap v442: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 945 op/s 2024-04-25T00:05:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:05 smithi173 ceph-mon[32402]: pgmap v442: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 945 op/s 2024-04-25T00:05:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 120.0 s, 822.6 tps, lat 121.685 ms stddev 18.990 2024-04-25T00:05:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:07 smithi188 ceph-mon[33285]: pgmap v443: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 967 op/s 2024-04-25T00:05:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:07 smithi110 ceph-mon[29187]: pgmap v443: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 967 op/s 2024-04-25T00:05:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:07 smithi173 ceph-mon[32402]: pgmap v443: 97 pgs: 97 active+clean; 1006 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 967 op/s 2024-04-25T00:05:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:09 smithi188 ceph-mon[33285]: pgmap v444: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 1.15k op/s 2024-04-25T00:05:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:09 smithi110 ceph-mon[29187]: pgmap v444: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 1.15k op/s 2024-04-25T00:05:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:09 smithi173 ceph-mon[32402]: pgmap v444: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 1.15k op/s 2024-04-25T00:05:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 125.0 s, 804.4 tps, lat 123.894 ms stddev 23.946 2024-04-25T00:05:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 -- 172.21.15.110:0/2029161262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0450102270 msgr2=0x7f0450108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 --2- 172.21.15.110:0/2029161262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0450102270 0x7f0450108b00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0440009900 tx=0x7f044002a410 comp rx=0 tx=0).stop 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 -- 172.21.15.110:0/2029161262 shutdown_connections 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 --2- 172.21.15.110:0/2029161262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0450102270 0x7f0450108b00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 --2- 172.21.15.110:0/2029161262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04501018b0 0x7f0450101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 --2- 172.21.15.110:0/2029161262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04501006c0 0x7f0450100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 -- 172.21.15.110:0/2029161262 >> 172.21.15.110:0/2029161262 conn(0x7f04500fbc30 msgr2=0x7f04500fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 -- 172.21.15.110:0/2029161262 shutdown_connections 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.835+0000 7f0455a15640 1 -- 172.21.15.110:0/2029161262 wait complete. 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f0455a15640 1 Processor -- start 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f0455a15640 1 -- start start 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f0455a15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04501006c0 0x7f04501a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f0455a15640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04501018b0 0x7f04501a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f0455a15640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0450102270 0x7f04501a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f0455a15640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f045010c1f0 con 0x7f04501018b0 2024-04-25T00:05:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f0455a15640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f045010c070 con 0x7f0450102270 2024-04-25T00:05:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f0455a15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f045010c370 con 0x7f04501006c0 2024-04-25T00:05:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f044e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04501018b0 0x7f04501a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f044effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04501006c0 0x7f04501a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f044f7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0450102270 0x7f04501a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f044effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04501006c0 0x7f04501a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:34876/0 (socket says 172.21.15.110:34876) 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f044e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04501018b0 0x7f04501a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41106/0 (socket says 172.21.15.110:41106) 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.836+0000 7f044effd640 1 -- 172.21.15.110:0/4015693445 learned_addr learned my addr 172.21.15.110:0/4015693445 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f044f7fe640 1 -- 172.21.15.110:0/4015693445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04501006c0 msgr2=0x7f04501a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f044f7fe640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04501006c0 0x7f04501a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f044f7fe640 1 -- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04501018b0 msgr2=0x7f04501a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f044f7fe640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04501018b0 0x7f04501a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f044f7fe640 1 -- 172.21.15.110:0/4015693445 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04501a5b60 con 0x7f0450102270 2024-04-25T00:05:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f044e7fc640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04501018b0 0x7f04501a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f044f7fe640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0450102270 0x7f04501a52a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0440031440 tx=0x7f0440004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f0454a13640 1 -- 172.21.15.110:0/4015693445 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044003f030 con 0x7f0450102270 2024-04-25T00:05:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f0455a15640 1 -- 172.21.15.110:0/4015693445 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04501a5df0 con 0x7f0450102270 2024-04-25T00:05:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f0454a13640 1 -- 172.21.15.110:0/4015693445 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0440003aa0 con 0x7f0450102270 2024-04-25T00:05:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f0454a13640 1 -- 172.21.15.110:0/4015693445 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440035070 con 0x7f0450102270 2024-04-25T00:05:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.837+0000 7f0455a15640 1 -- 172.21.15.110:0/4015693445 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501a6260 con 0x7f0450102270 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0454a13640 1 -- 172.21.15.110:0/4015693445 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0440042030 con 0x7f0450102270 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0458802740 1 Processor -- start 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0458802740 1 -- start start 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0458802740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3901270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0458802740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3906340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0454a13640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f043c0a2c30 0x7f043c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0458802740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 0x560bc3908e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0458802740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560bc3864cb0 con 0x560bc3901840 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0458802740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560bc380d5b0 con 0x560bc3900e50 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.840+0000 7f0458802740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560bc3860c70 con 0x560bc39069b0 2024-04-25T00:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044e7fc640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f043c0a2c30 0x7f043c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3906340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f0454a13640 1 -- 172.21.15.110:0/4015693445 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f04400ef560 con 0x7f0450102270 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3906340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41114/0 (socket says 172.21.15.110:41114) 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 -- 172.21.15.110:0/287380 learned_addr learned my addr 172.21.15.110:0/287380 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044e7fc640 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3901270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044f7fe640 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 0x560bc3908e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 -- 172.21.15.110:0/287380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 msgr2=0x560bc3908e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 0x560bc3908e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 -- 172.21.15.110:0/287380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 msgr2=0x560bc3901270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3901270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 -- 172.21.15.110:0/287380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc3579210 con 0x560bc3901840 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044effd640 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3906340 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0448011490 tx=0x7f0448011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4061006641b87d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f044e7fc640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f043c0a2c30 0x7f043c0a50f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0450108f80 tx=0x7f0438009c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.841+0000 7f0445ffb640 1 -- 172.21.15.110:0/287380 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448003b70 con 0x560bc3901840 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0445ffb640 1 -- 172.21.15.110:0/287380 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0448003d10 con 0x560bc3901840 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0445ffb640 1 -- 172.21.15.110:0/287380 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448012a50 con 0x560bc3901840 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 -- 172.21.15.110:0/287380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 msgr2=0x560bc3906340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3906340 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0448011490 tx=0x7f0448011950 comp rx=0 tx=0).stop 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 -- 172.21.15.110:0/287380 shutdown_connections 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 0x560bc3908e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3906340 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 --2- 172.21.15.110:0/287380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3901270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 -- 172.21.15.110:0/287380 >> 172.21.15.110:0/287380 conn(0x560bc38fc4c0 msgr2=0x560bc38fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 -- 172.21.15.110:0/287380 shutdown_connections 2024-04-25T00:05:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.842+0000 7f0458802740 1 -- 172.21.15.110:0/287380 wait complete. 2024-04-25T00:05:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.843+0000 7f0458802740 1 -- start start 2024-04-25T00:05:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.843+0000 7f0458802740 1 Processor -- start 2024-04-25T00:05:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.843+0000 7f0458802740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3900e50 0x560bc390b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.843+0000 7f0458802740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc390b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.843+0000 7f0458802740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc39069b0 0x560bc39127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.843+0000 7f0458802740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560bc3909960 con 0x560bc3901840 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.843+0000 7f0458802740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560bc3820b50 con 0x560bc39069b0 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.843+0000 7f0458802740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560bc3909ae0 con 0x560bc3900e50 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc390b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc390b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41130/0 (socket says 172.21.15.110:41130) 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 -- 172.21.15.110:0/743634427 learned_addr learned my addr 172.21.15.110:0/743634427 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044f7fe640 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc39069b0 0x560bc39127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f0458802740 10 client.0 _ll_register_callbacks cb 0x560bc385bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 -- 172.21.15.110:0/743634427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3900e50 msgr2=0x560bc390b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044effd640 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3900e50 0x560bc390b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3900e50 0x560bc390b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 -- 172.21.15.110:0/743634427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc39069b0 msgr2=0x560bc39127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc39069b0 0x560bc39127d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 -- 172.21.15.110:0/743634427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc3645620 con 0x560bc3901840 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f044e7fc640 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc390b790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f043800fc70 tx=0x7f0438004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c531b2b5720d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f042effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f042effd640 1 -- 172.21.15.110:0/743634427 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438003b70 con 0x560bc3901840 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f0458802740 1 -- 172.21.15.110:0/743634427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0448003d10 con 0x560bc3901840 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.844+0000 7f042effd640 1 -- 172.21.15.110:0/743634427 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0438003d10 con 0x560bc3901840 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f042effd640 1 -- 172.21.15.110:0/743634427 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0438029430 con 0x560bc3901840 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f0458802740 10 client.15513 fetch_fsmap learned FSMap version 27 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f0458802740 1 -- 172.21.15.110:0/743634427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bc3915bb0 con 0x560bc3901840 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f042effd640 1 -- 172.21.15.110:0/743634427 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0448003d10 con 0x560bc3901840 2024-04-25T00:05:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f042effd640 1 -- 172.21.15.110:0/743634427 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f043801f070 con 0x560bc3901840 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f0458802740 10 client.15513 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f0458802740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f0458802740 20 client.15513 populate_metadata read hostname 'smithi110' 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f0458802740 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560bc39177b0 0x560bc3919c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f0458802740 4 client.15513 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f0458802740 1 -- 172.21.15.110:0/743634427 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bc38721f0 con 0x560bc39177b0 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.845+0000 7f044effd640 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560bc39177b0 0x560bc3919c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.846+0000 7f042effd640 10 client.15513 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.846+0000 7f044effd640 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560bc39177b0 0x560bc3919c70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6edb88bc63f70a server_cookie=c053a7e2e71ac4bd in_seq=0 out_seq=0 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f042effd640 1 -- 172.21.15.110:0/743634427 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bc38721f0 con 0x560bc39177b0 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f042effd640 10 client.15513 handle_command_reply: tid=0 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 client.15513 shutdown 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 -- 172.21.15.110:0/743634427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 msgr2=0x560bc390b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc390b790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f043800fc70 tx=0x7f0438004730 comp rx=0 tx=0).stop 2024-04-25T00:05:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 -- 172.21.15.110:0/743634427 shutdown_connections 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560bc39177b0 0x560bc3919c70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc39069b0 0x560bc39127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f042effd640 0 client.15513 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc390b790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 --2- 172.21.15.110:0/743634427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3900e50 0x560bc390b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 -- 172.21.15.110:0/743634427 >> 172.21.15.110:0/743634427 conn(0x560bc38fc4c0 msgr2=0x560bc38fb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 -- 172.21.15.110:0/743634427 shutdown_connections 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 1 -- 172.21.15.110:0/743634427 wait complete. 2024-04-25T00:05:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.847+0000 7f0458802740 20 client.15513 trim_cache size 0 max 16384 2024-04-25T00:05:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f0458802740 1 Processor -- start 2024-04-25T00:05:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f0458802740 1 -- start start 2024-04-25T00:05:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f0458802740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc38ff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f0458802740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3912790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f0458802740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 0x560bc38ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f0458802740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04501b1960 con 0x560bc3901840 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f0458802740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04501af1a0 con 0x560bc3900e50 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f0458802740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04501acff0 con 0x560bc39069b0 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f044e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3912790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f044effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc38ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f044e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3912790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41146/0 (socket says 172.21.15.110:41146) 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f044effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc38ff470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:56662/0 (socket says 172.21.15.110:56662) 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f044e7fc640 1 -- 172.21.15.110:0/2261404651 learned_addr learned my addr 172.21.15.110:0/2261404651 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.867+0000 7f044f7fe640 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 0x560bc38ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f044effd640 1 -- 172.21.15.110:0/2261404651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 msgr2=0x560bc38ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f044effd640 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 0x560bc38ff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f044effd640 1 -- 172.21.15.110:0/2261404651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 msgr2=0x560bc3912790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f044effd640 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3912790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f044effd640 1 -- 172.21.15.110:0/2261404651 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc38fc000 con 0x560bc3900e50 2024-04-25T00:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f044e7fc640 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3912790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f044effd640 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc38ff470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x560bc38fd430 tx=0x7f0448003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53472380f9bca24f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f04457fa640 1 -- 172.21.15.110:0/2261404651 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448003be0 con 0x560bc3900e50 2024-04-25T00:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f04457fa640 1 -- 172.21.15.110:0/2261404651 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0448020040 con 0x560bc3900e50 2024-04-25T00:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.868+0000 7f04457fa640 1 -- 172.21.15.110:0/2261404651 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448005070 con 0x560bc3900e50 2024-04-25T00:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 -- 172.21.15.110:0/2261404651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 msgr2=0x560bc38ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc38ff470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x560bc38fd430 tx=0x7f0448003970 comp rx=0 tx=0).stop 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 -- 172.21.15.110:0/2261404651 shutdown_connections 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc39069b0 0x560bc38ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc3901840 0x560bc3912790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 --2- 172.21.15.110:0/2261404651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc38ff470 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 -- 172.21.15.110:0/2261404651 >> 172.21.15.110:0/2261404651 conn(0x560bc38fc4c0 msgr2=0x560bc38fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 -- 172.21.15.110:0/2261404651 shutdown_connections 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.869+0000 7f0458802740 1 -- 172.21.15.110:0/2261404651 wait complete. 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 1 -- start start 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 1 Processor -- start 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3910100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3901840 0x560bc3910640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc39069b0 0x560bc3911600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04501ad660 con 0x560bc39069b0 2024-04-25T00:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04501aead0 con 0x560bc3900e50 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04501add30 con 0x560bc3901840 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f0458802740 10 client.0 _ll_register_callbacks cb 0x7f0438009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f044effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3910100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f044f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc39069b0 0x560bc3911600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f044effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3910100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:56670/0 (socket says 172.21.15.110:56670) 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f044effd640 1 -- 172.21.15.110:0/1809290451 learned_addr learned my addr 172.21.15.110:0/1809290451 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.870+0000 7f044e7fc640 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3901840 0x560bc3910640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f044effd640 1 -- 172.21.15.110:0/1809290451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3901840 msgr2=0x560bc3910640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f044effd640 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3901840 0x560bc3910640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f044effd640 1 -- 172.21.15.110:0/1809290451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc39069b0 msgr2=0x560bc3911600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f044effd640 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc39069b0 0x560bc3911600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f044effd640 1 -- 172.21.15.110:0/1809290451 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc3911bf0 con 0x560bc3900e50 2024-04-25T00:05:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f044effd640 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3910100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04480039e0 tx=0x7f0448009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1f51c285e7e0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f042effd640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f042effd640 1 -- 172.21.15.110:0/1809290451 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044802d040 con 0x560bc3900e50 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f042effd640 1 -- 172.21.15.110:0/1809290451 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0448012d30 con 0x560bc3900e50 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f0458802740 1 -- 172.21.15.110:0/1809290451 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0448020040 con 0x560bc3900e50 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.871+0000 7f042effd640 1 -- 172.21.15.110:0/1809290451 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0448003be0 con 0x560bc3900e50 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f0458802740 10 client.25100 fetch_fsmap learned FSMap version 27 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f0458802740 1 -- 172.21.15.110:0/1809290451 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560bc397dfa0 con 0x560bc3900e50 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f042effd640 1 -- 172.21.15.110:0/1809290451 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0448020040 con 0x560bc3900e50 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f042effd640 1 -- 172.21.15.110:0/1809290451 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f044800f040 con 0x560bc3900e50 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f0458802740 10 client.25100 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f0458802740 10 client.25100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f0458802740 20 client.25100 populate_metadata read hostname 'smithi110' 2024-04-25T00:05:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f0458802740 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560bc397e7f0 0x560bc3980c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f0458802740 4 client.25100 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:05:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f0458802740 1 -- 172.21.15.110:0/1809290451 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bc386c540 con 0x560bc397e7f0 2024-04-25T00:05:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.872+0000 7f044e7fc640 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560bc397e7f0 0x560bc3980c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.873+0000 7f042effd640 10 client.25100 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.873+0000 7f044e7fc640 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560bc397e7f0 0x560bc3980c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4839bb32eb7c866d server_cookie=e3881673e8d75369 in_seq=0 out_seq=0 2024-04-25T00:05:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.889+0000 7f042effd640 1 -- 172.21.15.110:0/1809290451 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bc386c540 con 0x560bc397e7f0 2024-04-25T00:05:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.889+0000 7f042effd640 10 client.25100 handle_command_reply: tid=0 2024-04-25T00:05:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.889+0000 7f0458802740 1 client.25100 shutdown 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 -- 172.21.15.110:0/1809290451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 msgr2=0x560bc3910100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3910100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04480039e0 tx=0x7f0448009520 comp rx=0 tx=0).stop 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 -- 172.21.15.110:0/1809290451 shutdown_connections 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560bc397e7f0 0x560bc3980c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560bc39069b0 0x560bc3911600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bc3901840 0x560bc3910640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f042effd640 0 client.25100 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 --2- 172.21.15.110:0/1809290451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bc3900e50 0x560bc3910100 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 -- 172.21.15.110:0/1809290451 >> 172.21.15.110:0/1809290451 conn(0x560bc38fc4c0 msgr2=0x560bc38ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 -- 172.21.15.110:0/1809290451 shutdown_connections 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 1 -- 172.21.15.110:0/1809290451 wait complete. 2024-04-25T00:05:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.890+0000 7f0458802740 20 client.25100 trim_cache size 0 max 16384 2024-04-25T00:05:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:05:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:05:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:05:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 -- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f043c0a2c30 msgr2=0x7f043c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f043c0a2c30 0x7f043c0a50f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0450108f80 tx=0x7f0438009c90 comp rx=0 tx=0).stop 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 -- 172.21.15.110:0/4015693445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0450102270 msgr2=0x7f04501a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0450102270 0x7f04501a52a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0440031440 tx=0x7f0440004270 comp rx=0 tx=0).stop 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 -- 172.21.15.110:0/4015693445 shutdown_connections 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f043c0a2c30 0x7f043c0a50f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0450102270 0x7f04501a52a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04501018b0 0x7f04501a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 --2- 172.21.15.110:0/4015693445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04501006c0 0x7f04501a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 -- 172.21.15.110:0/4015693445 >> 172.21.15.110:0/4015693445 conn(0x7f04500fbc30 msgr2=0x7f0450069f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 -- 172.21.15.110:0/4015693445 shutdown_connections 2024-04-25T00:05:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:11.891+0000 7f0455a15640 1 -- 172.21.15.110:0/4015693445 wait complete. 2024-04-25T00:05:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:11 smithi188 ceph-mon[33285]: pgmap v445: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 979 op/s 2024-04-25T00:05:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:11 smithi110 ceph-mon[29187]: pgmap v445: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 979 op/s 2024-04-25T00:05:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:11 smithi173 ceph-mon[32402]: pgmap v445: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 979 op/s 2024-04-25T00:05:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:13 smithi188 ceph-mon[33285]: pgmap v446: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 955 op/s 2024-04-25T00:05:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:13 smithi110 ceph-mon[29187]: pgmap v446: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 955 op/s 2024-04-25T00:05:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:13 smithi173 ceph-mon[32402]: pgmap v446: 97 pgs: 97 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 MiB/s wr, 955 op/s 2024-04-25T00:05:16.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:15 smithi188 ceph-mon[33285]: pgmap v447: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:05:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:15 smithi110 ceph-mon[29187]: pgmap v447: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:05:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:15 smithi173 ceph-mon[32402]: pgmap v447: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:05:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 130.0 s, 847.0 tps, lat 118.377 ms stddev 13.559 2024-04-25T00:05:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:17 smithi188 ceph-mon[33285]: pgmap v448: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:05:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:17 smithi110 ceph-mon[29187]: pgmap v448: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:05:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:17 smithi173 ceph-mon[32402]: pgmap v448: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:05:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:19 smithi188 ceph-mon[33285]: pgmap v449: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 1.09k op/s 2024-04-25T00:05:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:19 smithi110 ceph-mon[29187]: pgmap v449: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 1.09k op/s 2024-04-25T00:05:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:19 smithi173 ceph-mon[32402]: pgmap v449: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 1.09k op/s 2024-04-25T00:05:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 135.0 s, 878.8 tps, lat 113.644 ms stddev 23.975 2024-04-25T00:05:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:21 smithi188 ceph-mon[33285]: pgmap v450: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-04-25T00:05:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:21 smithi110 ceph-mon[29187]: pgmap v450: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-04-25T00:05:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:21 smithi173 ceph-mon[32402]: pgmap v450: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-04-25T00:05:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:23 smithi188 ceph-mon[33285]: pgmap v451: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 1.13k op/s 2024-04-25T00:05:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:23 smithi110 ceph-mon[29187]: pgmap v451: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 1.13k op/s 2024-04-25T00:05:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:23 smithi173 ceph-mon[32402]: pgmap v451: 97 pgs: 97 active+clean; 1007 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 1.13k op/s 2024-04-25T00:05:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:24 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.025+0000 7f1feb458640 1 -- 172.21.15.110:0/3781899466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 msgr2=0x7f1fdc0962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.025+0000 7f1feb458640 1 --2- 172.21.15.110:0/3781899466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 0x7f1fdc0962f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009900 tx=0x7f1fd402a410 comp rx=0 tx=0).stop 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.025+0000 7f1feb458640 1 -- 172.21.15.110:0/3781899466 shutdown_connections 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.025+0000 7f1feb458640 1 --2- 172.21.15.110:0/3781899466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc096830 0x7f1fdc09d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.025+0000 7f1feb458640 1 --2- 172.21.15.110:0/3781899466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 0x7f1fdc0962f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.025+0000 7f1feb458640 1 --2- 172.21.15.110:0/3781899466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fdc094c80 0x7f1fdc0950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.025+0000 7f1feb458640 1 -- 172.21.15.110:0/3781899466 >> 172.21.15.110:0/3781899466 conn(0x7f1fdc0901d0 msgr2=0x7f1fdc092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.025+0000 7f1feb458640 1 -- 172.21.15.110:0/3781899466 shutdown_connections 2024-04-25T00:05:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.026+0000 7f1feb458640 1 -- 172.21.15.110:0/3781899466 wait complete. 2024-04-25T00:05:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.026+0000 7f1feb458640 1 Processor -- start 2024-04-25T00:05:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.026+0000 7f1feb458640 1 -- start start 2024-04-25T00:05:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.026+0000 7f1feb458640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fdc094c80 0x7f1fdc005820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.026+0000 7f1feb458640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 0x7f1fdc130cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.026+0000 7f1feb458640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc096830 0x7f1fdc1332f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1feb458640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc0a07b0 con 0x7f1fdc094c80 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1feb458640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc0a0630 con 0x7f1fdc096830 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1feb458640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc0a0930 con 0x7f1fdc095e70 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fea456640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fdc094c80 0x7f1fdc005820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 0x7f1fdc130cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 0x7f1fdc130cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:53416/0 (socket says 172.21.15.110:53416) 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 -- 172.21.15.110:0/2634783892 learned_addr learned my addr 172.21.15.110:0/2634783892 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1feac57640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc096830 0x7f1fdc1332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 -- 172.21.15.110:0/2634783892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc096830 msgr2=0x7f1fdc1332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc096830 0x7f1fdc1332f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 -- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fdc094c80 msgr2=0x7f1fdc005820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fdc094c80 0x7f1fdc005820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 -- 172.21.15.110:0/2634783892 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc12ab20 con 0x7f1fdc095e70 2024-04-25T00:05:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.027+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 0x7f1fdc130cd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005940 tx=0x7f1fd403eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.028+0000 7f1fdb7fe640 1 -- 172.21.15.110:0/2634783892 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd404c070 con 0x7f1fdc095e70 2024-04-25T00:05:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.028+0000 7f1fdb7fe640 1 -- 172.21.15.110:0/2634783892 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd40318f0 con 0x7f1fdc095e70 2024-04-25T00:05:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.028+0000 7f1fdb7fe640 1 -- 172.21.15.110:0/2634783892 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4035050 con 0x7f1fdc095e70 2024-04-25T00:05:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.028+0000 7f1feb458640 1 -- 172.21.15.110:0/2634783892 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc12ad50 con 0x7f1fdc095e70 2024-04-25T00:05:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.028+0000 7f1feb458640 1 -- 172.21.15.110:0/2634783892 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc12b1f0 con 0x7f1fdc095e70 2024-04-25T00:05:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.031+0000 7f1fdb7fe640 1 -- 172.21.15.110:0/2634783892 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1fd404f070 con 0x7f1fdc095e70 2024-04-25T00:05:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.032+0000 7f1fef3c8740 1 Processor -- start 2024-04-25T00:05:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.032+0000 7f1fef3c8740 1 -- start start 2024-04-25T00:05:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b873e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b8c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b8efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c768aeacb0 con 0x55c768b8cb20 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c768a935b0 con 0x55c768b86fc0 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c768ae6c70 con 0x55c768b879b0 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fea456640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b8c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1feac57640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b873e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1feac57640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b873e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:38946/0 (socket says 172.21.15.110:38946) 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1feac57640 1 -- 172.21.15.110:0/492878757 learned_addr learned my addr 172.21.15.110:0/492878757 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b8efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fea456640 1 -- 172.21.15.110:0/492878757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 msgr2=0x55c768b873e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fea456640 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b873e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fea456640 1 -- 172.21.15.110:0/492878757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 msgr2=0x55c768b8efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fea456640 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b8efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fea456640 1 -- 172.21.15.110:0/492878757 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7687ff210 con 0x55c768b879b0 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.033+0000 7f1fdb7fe640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1fd00a2ce0 0x7f1fd00a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.034+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b8efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.034+0000 7f1feac57640 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b873e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.034+0000 7f1fea456640 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b8c4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009bf0 tx=0x7f1fe0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63993a1c3308ce80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.034+0000 7f1feac57640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1fd00a2ce0 0x7f1fd00a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.034+0000 7f1fd8ff9640 1 -- 172.21.15.110:0/492878757 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0004050 con 0x55c768b879b0 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.034+0000 7f1fdb7fe640 1 -- 172.21.15.110:0/2634783892 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1fd40f7200 con 0x7f1fdc095e70 2024-04-25T00:05:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.034+0000 7f1fd8ff9640 1 -- 172.21.15.110:0/492878757 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fe00054d0 con 0x55c768b879b0 2024-04-25T00:05:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.034+0000 7f1fd8ff9640 1 -- 172.21.15.110:0/492878757 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0004ca0 con 0x55c768b879b0 2024-04-25T00:05:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 -- 172.21.15.110:0/492878757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 msgr2=0x55c768b8c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b8c4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009bf0 tx=0x7f1fe0011810 comp rx=0 tx=0).stop 2024-04-25T00:05:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1feac57640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1fd00a2ce0 0x7f1fd00a51a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40648b0 tx=0x7f1fe4062cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 -- 172.21.15.110:0/492878757 shutdown_connections 2024-04-25T00:05:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b8efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b8c4b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/492878757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b873e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 -- 172.21.15.110:0/492878757 >> 172.21.15.110:0/492878757 conn(0x55c768b82630 msgr2=0x55c768b82a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 -- 172.21.15.110:0/492878757 shutdown_connections 2024-04-25T00:05:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.035+0000 7f1fef3c8740 1 -- 172.21.15.110:0/492878757 wait complete. 2024-04-25T00:05:25.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:24 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.036+0000 7f1fef3c8740 1 -- start start 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.036+0000 7f1fef3c8740 1 Processor -- start 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.037+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.037+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b91910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.037+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b98950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.037+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c768b8fb70 con 0x55c768b8cb20 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.037+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c768aa6b50 con 0x55c768b86fc0 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.037+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c768b8fcf0 con 0x55c768b879b0 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fef3c8740 10 client.0 _ll_register_callbacks cb 0x55c768ae1ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1feac57640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b98950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b913d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:38958/0 (socket says 172.21.15.110:38958) 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1feac57640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b98950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43710/0 (socket says 172.21.15.110:43710) 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 -- 172.21.15.110:0/3022793646 learned_addr learned my addr 172.21.15.110:0/3022793646 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b91910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 -- 172.21.15.110:0/3022793646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 msgr2=0x55c768b91910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b91910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 -- 172.21.15.110:0/3022793646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 msgr2=0x55c768b98950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b98950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 -- 172.21.15.110:0/3022793646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7688cb620 con 0x55c768b86fc0 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b91910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fea456640 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b913d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1fe001b040 tx=0x7f1fe0009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13b93af57cb70159 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.038+0000 7f1fcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:05:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.039+0000 7f1fcdffb640 1 -- 172.21.15.110:0/3022793646 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe003a070 con 0x55c768b86fc0 2024-04-25T00:05:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.039+0000 7f1fcdffb640 1 -- 172.21.15.110:0/3022793646 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1fe0003a00 con 0x55c768b86fc0 2024-04-25T00:05:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.039+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3022793646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe00054d0 con 0x55c768b86fc0 2024-04-25T00:05:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.039+0000 7f1fcdffb640 1 -- 172.21.15.110:0/3022793646 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fe0046050 con 0x55c768b86fc0 2024-04-25T00:05:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.040+0000 7f1fef3c8740 10 client.25109 fetch_fsmap learned FSMap version 28 2024-04-25T00:05:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.040+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3022793646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c768b9bc80 con 0x55c768b86fc0 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.040+0000 7f1fcdffb640 1 -- 172.21.15.110:0/3022793646 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fe00054d0 con 0x55c768b86fc0 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.041+0000 7f1fcdffb640 1 -- 172.21.15.110:0/3022793646 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1fe0034440 con 0x55c768b86fc0 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.041+0000 7f1fef3c8740 10 client.25109 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.041+0000 7f1fef3c8740 10 client.25109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.041+0000 7f1fef3c8740 20 client.25109 populate_metadata read hostname 'smithi110' 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.041+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c768b9d170 0x55c768b9f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.041+0000 7f1fef3c8740 4 client.25109 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.041+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3022793646 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c768af81f0 con 0x55c768b9d170 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.041+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c768b9d170 0x55c768b9f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.042+0000 7f1fcdffb640 10 client.25109 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.042+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c768b9d170 0x55c768b9f5f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f691019ff9c3274d server_cookie=fd8d94a7be02c6c8 in_seq=0 out_seq=0 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.042+0000 7f1fcdffb640 1 -- 172.21.15.110:0/3022793646 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c768af81f0 con 0x55c768b9d170 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.042+0000 7f1fcdffb640 10 client.25109 handle_command_reply: tid=0 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.042+0000 7f1fef3c8740 1 client.25109 shutdown 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3022793646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 msgr2=0x55c768b913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b913d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1fe001b040 tx=0x7f1fe0009fd0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3022793646 shutdown_connections 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c768b9d170 0x55c768b9f5f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b98950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b91910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3022793646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b86fc0 0x55c768b913d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3022793646 >> 172.21.15.110:0/3022793646 conn(0x55c768b82630 msgr2=0x55c768b81830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fcdffb640 0 client.25109 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3022793646 shutdown_connections 2024-04-25T00:05:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3022793646 wait complete. 2024-04-25T00:05:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.043+0000 7f1fef3c8740 20 client.25109 trim_cache size 0 max 16384 2024-04-25T00:05:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.070+0000 7f1fef3c8740 1 Processor -- start 2024-04-25T00:05:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fef3c8740 1 -- start start 2024-04-25T00:05:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b86fc0 0x55c768b85dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b8cb20 0x55c768b90f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc13d7a0 con 0x55c768b86fc0 2024-04-25T00:05:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc13afe0 con 0x55c768b8cb20 2024-04-25T00:05:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc138e30 con 0x55c768b879b0 2024-04-25T00:05:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fea456640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b86fc0 0x55c768b85dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fe9c55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fea456640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b86fc0 0x55c768b85dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43724/0 (socket says 172.21.15.110:43724) 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1feac57640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b8cb20 0x55c768b90f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fea456640 1 -- 172.21.15.110:0/3276398538 learned_addr learned my addr 172.21.15.110:0/3276398538 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fea456640 1 -- 172.21.15.110:0/3276398538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 msgr2=0x55c768b909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fea456640 1 --2- 172.21.15.110:0/3276398538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fea456640 1 -- 172.21.15.110:0/3276398538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b8cb20 msgr2=0x55c768b90f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fea456640 1 --2- 172.21.15.110:0/3276398538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b8cb20 0x55c768b90f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1fea456640 1 -- 172.21.15.110:0/3276398538 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c768b83080 con 0x55c768b86fc0 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.071+0000 7f1feac57640 1 --2- 172.21.15.110:0/3276398538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b8cb20 0x55c768b90f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.072+0000 7f1fea456640 1 --2- 172.21.15.110:0/3276398538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b86fc0 0x55c768b85dd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0000c00 tx=0x7f1fe0009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444ce49c97377765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.072+0000 7f1fcffff640 1 -- 172.21.15.110:0/3276398538 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe003a050 con 0x55c768b86fc0 2024-04-25T00:05:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.072+0000 7f1fcffff640 1 -- 172.21.15.110:0/3276398538 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fe00043d0 con 0x55c768b86fc0 2024-04-25T00:05:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.072+0000 7f1fcffff640 1 -- 172.21.15.110:0/3276398538 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe003f440 con 0x55c768b86fc0 2024-04-25T00:05:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3276398538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b86fc0 msgr2=0x55c768b85dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3276398538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b86fc0 0x55c768b85dd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0000c00 tx=0x7f1fe0009570 comp rx=0 tx=0).stop 2024-04-25T00:05:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3276398538 shutdown_connections 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3276398538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b8cb20 0x55c768b90f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3276398538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b879b0 0x55c768b909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/3276398538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b86fc0 0x55c768b85dd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3276398538 >> 172.21.15.110:0/3276398538 conn(0x55c768b82630 msgr2=0x55c768b856c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3276398538 shutdown_connections 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.073+0000 7f1fef3c8740 1 -- 172.21.15.110:0/3276398538 wait complete. 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 1 -- start start 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 1 Processor -- start 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b86fc0 0x55c768b954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b879b0 0x55c768b95a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1394a0 con 0x55c768b8cb20 2024-04-25T00:05:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fea456640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b86fc0 0x55c768b954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc13a910 con 0x55c768b879b0 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc139b70 con 0x55c768b86fc0 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fea456640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b86fc0 0x55c768b954c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:53464/0 (socket says 172.21.15.110:53464) 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fea456640 1 -- 172.21.15.110:0/4061085984 learned_addr learned my addr 172.21.15.110:0/4061085984 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b879b0 0x55c768b95a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.074+0000 7f1fef3c8740 10 client.0 _ll_register_callbacks cb 0x7f1fe4050410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1feac57640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fe9c55640 1 -- 172.21.15.110:0/4061085984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b86fc0 msgr2=0x55c768b954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b86fc0 0x55c768b954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fe9c55640 1 -- 172.21.15.110:0/4061085984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 msgr2=0x55c768b96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fe9c55640 1 -- 172.21.15.110:0/4061085984 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c768b97000 con 0x55c768b879b0 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fea456640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b86fc0 0x55c768b954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1feac57640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fe9c55640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b879b0 0x55c768b95a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40f4d80 tx=0x7f1fd40c5150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5b47bfe61bb7f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fcdffb640 1 -- 172.21.15.110:0/4061085984 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40c8d40 con 0x55c768b879b0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fcdffb640 1 -- 172.21.15.110:0/4061085984 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f1fd40f4db0 con 0x55c768b879b0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fef3c8740 1 -- 172.21.15.110:0/4061085984 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe00043d0 con 0x55c768b879b0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fcdffb640 1 -- 172.21.15.110:0/4061085984 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fd40c1000 con 0x55c768b879b0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fef3c8740 10 client.25115 fetch_fsmap learned FSMap version 28 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.075+0000 7f1fef3c8740 1 -- 172.21.15.110:0/4061085984 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c768c03730 con 0x55c768b879b0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fcdffb640 1 -- 172.21.15.110:0/4061085984 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fe00043d0 con 0x55c768b879b0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fcdffb640 1 -- 172.21.15.110:0/4061085984 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1fd40cd200 con 0x55c768b879b0 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fef3c8740 10 client.25115 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fef3c8740 10 client.25115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fef3c8740 20 client.25115 populate_metadata read hostname 'smithi110' 2024-04-25T00:05:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c768c04000 0x55c768c06420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fef3c8740 4 client.25115 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:05:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fef3c8740 1 -- 172.21.15.110:0/4061085984 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c768af2540 con 0x55c768c04000 2024-04-25T00:05:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.076+0000 7f1fea456640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c768c04000 0x55c768c06420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.077+0000 7f1fea456640 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c768c04000 0x55c768c06420 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76d0f28944b88ab server_cookie=24841f62aa4122ba in_seq=0 out_seq=0 2024-04-25T00:05:25.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.077+0000 7f1fcdffb640 10 client.25115 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fcdffb640 1 -- 172.21.15.110:0/4061085984 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c768af2540 con 0x55c768c04000 2024-04-25T00:05:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fcdffb640 10 client.25115 handle_command_reply: tid=0 2024-04-25T00:05:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 client.25115 shutdown 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 -- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b879b0 msgr2=0x55c768b95a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b879b0 0x55c768b95a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40f4d80 tx=0x7f1fd40c5150 comp rx=0 tx=0).stop 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 -- 172.21.15.110:0/4061085984 shutdown_connections 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55c768c04000 0x55c768c06420 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c768b8cb20 0x55c768b96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fcdffb640 0 client.25115 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c768b879b0 0x55c768b95a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 --2- 172.21.15.110:0/4061085984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c768b86fc0 0x55c768b954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.097+0000 7f1fef3c8740 1 -- 172.21.15.110:0/4061085984 >> 172.21.15.110:0/4061085984 conn(0x55c768b82630 msgr2=0x55c768b80fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.098+0000 7f1fef3c8740 1 -- 172.21.15.110:0/4061085984 shutdown_connections 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.098+0000 7f1fef3c8740 1 -- 172.21.15.110:0/4061085984 wait complete. 2024-04-25T00:05:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.098+0000 7f1fef3c8740 20 client.25115 trim_cache size 0 max 16384 2024-04-25T00:05:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:05:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:05:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:05:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:05:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 -- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1fd00a2ce0 msgr2=0x7f1fd00a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1fd00a2ce0 0x7f1fd00a51a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40648b0 tx=0x7f1fe4062cc0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 -- 172.21.15.110:0/2634783892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 msgr2=0x7f1fdc130cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 0x7f1fdc130cd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005940 tx=0x7f1fd403eca0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 -- 172.21.15.110:0/2634783892 shutdown_connections 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f1fd00a2ce0 0x7f1fd00a51a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc096830 0x7f1fdc1332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fdc095e70 0x7f1fdc130cd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 --2- 172.21.15.110:0/2634783892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fdc094c80 0x7f1fdc005820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 -- 172.21.15.110:0/2634783892 >> 172.21.15.110:0/2634783892 conn(0x7f1fdc0901d0 msgr2=0x7f1fdc0991f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.099+0000 7f1feb458640 1 -- 172.21.15.110:0/2634783892 shutdown_connections 2024-04-25T00:05:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:25.100+0000 7f1feb458640 1 -- 172.21.15.110:0/2634783892 wait complete. 2024-04-25T00:05:25.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:24 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:25 smithi188 ceph-mon[33285]: pgmap v452: 97 pgs: 97 active+clean; 1008 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 1.16k op/s 2024-04-25T00:05:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:25 smithi110 ceph-mon[29187]: pgmap v452: 97 pgs: 97 active+clean; 1008 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 1.16k op/s 2024-04-25T00:05:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:25 smithi173 ceph-mon[32402]: pgmap v452: 97 pgs: 97 active+clean; 1008 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.6 MiB/s wr, 1.16k op/s 2024-04-25T00:05:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 140.0 s, 835.2 tps, lat 120.053 ms stddev 18.978 2024-04-25T00:05:27.537 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:05:27 smithi110 sudo[85046]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:27.537 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:05:27 smithi110 sudo[85046]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:27.537 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:05:27 smithi110 sudo[85046]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:27.537 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:05:27 smithi110 sudo[85046]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:27.968 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:27 smithi173 ceph-mon[32402]: pgmap v453: 97 pgs: 97 active+clean; 1008 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 MiB/s wr, 898 op/s 2024-04-25T00:05:27.968 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:27 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:27 smithi188 ceph-mon[33285]: pgmap v453: 97 pgs: 97 active+clean; 1008 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 MiB/s wr, 898 op/s 2024-04-25T00:05:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:27 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:27 smithi110 ceph-mon[29187]: pgmap v453: 97 pgs: 97 active+clean; 1008 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 MiB/s wr, 898 op/s 2024-04-25T00:05:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:27 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:28.037 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:05:27 smithi110 sudo[85050]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:28.037 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:05:27 smithi110 sudo[85050]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:28.037 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:05:27 smithi110 sudo[85050]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:28.037 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:05:27 smithi110 sudo[85050]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:28.254 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:05:27 smithi173 sudo[65122]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:28.254 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:05:27 smithi173 sudo[65122]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:28.254 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:05:27 smithi173 sudo[65122]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:28.254 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:05:27 smithi173 sudo[65122]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:28.602 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:05:28 smithi173 sudo[65125]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:28.602 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:05:28 smithi173 sudo[65125]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:28.602 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:05:28 smithi173 sudo[65125]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:28.602 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:05:28 smithi173 sudo[65125]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:28.914 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:05:28 smithi188 sudo[66157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:28.914 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:05:28 smithi188 sudo[66157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:28.914 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:05:28 smithi188 sudo[66157]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:28.914 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:05:28 smithi188 sudo[66157]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:29.267 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:05:28 smithi188 sudo[66160]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:29.268 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:05:28 smithi188 sudo[66160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:29.268 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:05:28 smithi188 sudo[66160]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:29.268 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:05:28 smithi188 sudo[66160]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:29.536 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:05:29 smithi110 sudo[85053]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:29.536 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:05:29 smithi110 sudo[85053]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:29.537 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:05:29 smithi110 sudo[85053]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:29.537 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:05:29 smithi110 sudo[85053]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:29.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:29.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:29 smithi110 ceph-mon[29187]: pgmap v454: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-04-25T00:05:29.537 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:29 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:29 smithi173 ceph-mon[32402]: pgmap v454: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-04-25T00:05:29.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:29 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:29 smithi188 ceph-mon[33285]: pgmap v454: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-04-25T00:05:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:29 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:30.036 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:05:29 smithi110 sudo[85056]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:30.036 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:05:29 smithi110 sudo[85056]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:30.036 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:05:29 smithi110 sudo[85056]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:30.036 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:05:29 smithi110 sudo[85056]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:30.192 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:05:29 smithi173 sudo[65128]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:30.193 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:05:29 smithi173 sudo[65128]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:30.193 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:05:29 smithi173 sudo[65128]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:30.193 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:05:29 smithi173 sudo[65128]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:30.602 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:05:30 smithi173 sudo[65131]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:30.603 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:05:30 smithi173 sudo[65131]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:30.603 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:05:30 smithi173 sudo[65131]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:30.603 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:05:30 smithi173 sudo[65131]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:30.767 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:05:30 smithi188 sudo[66165]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:30.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:05:30 smithi188 sudo[66165]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:30.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:05:30 smithi188 sudo[66165]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:30.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:05:30 smithi188 sudo[66165]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:31.267 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:05:30 smithi188 sudo[66168]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:31.268 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:05:30 smithi188 sudo[66168]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:31.268 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:05:30 smithi188 sudo[66168]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:31.268 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:05:30 smithi188 sudo[66168]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:31.527 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:05:31 smithi110 sudo[85125]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:31.527 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:05:31 smithi110 sudo[85125]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:31.527 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:05:31 smithi110 sudo[85125]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:31.527 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:05:31 smithi110 sudo[85125]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 145.0 s, 803.0 tps, lat 123.787 ms stddev 17.385 2024-04-25T00:05:31.786 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:05:31 smithi110 sudo[85128]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:31.786 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:05:31 smithi110 sudo[85128]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:31.786 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:05:31 smithi110 sudo[85128]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:31.787 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:05:31 smithi110 sudo[85128]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:31.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:31 smithi110 ceph-mon[29187]: pgmap v455: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.8 MiB/s wr, 1.05k op/s 2024-04-25T00:05:31.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:31 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:05:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:31 smithi188 ceph-mon[33285]: pgmap v455: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.8 MiB/s wr, 1.05k op/s 2024-04-25T00:05:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:31 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:05:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:31 smithi173 ceph-mon[32402]: pgmap v455: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.8 MiB/s wr, 1.05k op/s 2024-04-25T00:05:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:31 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:05:32.102 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:05:31 smithi173 sudo[65230]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:32.103 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:05:31 smithi173 sudo[65230]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:32.103 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:05:31 smithi173 sudo[65230]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:32.103 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:05:31 smithi173 sudo[65230]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:32.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:05:32 smithi173 sudo[65266]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:32.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:05:32 smithi173 sudo[65266]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:32.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:05:32 smithi173 sudo[65266]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:32.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:05:32 smithi173 sudo[65266]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:32.722 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:05:32 smithi188 sudo[66301]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:32.722 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:05:32 smithi188 sudo[66301]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:32.722 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:05:32 smithi188 sudo[66301]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:32.722 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:05:32 smithi188 sudo[66301]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:32.723 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:05:32.723 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:05:32.723 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:33.017 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:05:32 smithi188 sudo[66305]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:33.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:05:32 smithi188 sudo[66305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:33.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:05:32 smithi188 sudo[66305]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:33.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:05:32 smithi188 sudo[66305]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:33.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:05:33.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:05:33.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:33.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:05:33.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:05:33.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:05:33.372 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:05:33 smithi110 sudo[85196]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:33.372 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:05:33 smithi110 sudo[85196]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:33.372 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:05:33 smithi110 sudo[85196]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:33.373 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:05:33 smithi110 sudo[85196]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:33.373 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:05:33 smithi110 sudo[85199]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:33.645 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:05:33 smithi110 sudo[85199]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:33.645 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:05:33 smithi110 sudo[85199]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:33.646 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:05:33 smithi110 sudo[85199]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:33.951 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:33 smithi173 ceph-mon[32402]: pgmap v456: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-04-25T00:05:33.952 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:05:33 smithi173 sudo[65270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:33.952 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:05:33 smithi173 sudo[65270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:33.952 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:05:33 smithi173 sudo[65270]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:33.952 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:05:33 smithi173 sudo[65270]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:33 smithi188 ceph-mon[33285]: pgmap v456: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-04-25T00:05:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:33 smithi110 ceph-mon[29187]: pgmap v456: 97 pgs: 97 active+clean; 1008 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-04-25T00:05:34.352 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:05:33 smithi173 sudo[65273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:34.352 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:05:33 smithi173 sudo[65273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:34.352 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:05:33 smithi173 sudo[65273]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:34.353 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:05:33 smithi173 sudo[65273]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:34.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:05:34 smithi188 sudo[66310]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:34.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:05:34 smithi188 sudo[66310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:34.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:05:34 smithi188 sudo[66310]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:34.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:05:34 smithi188 sudo[66310]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:35.017 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:05:34 smithi188 sudo[66313]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:05:34 smithi188 sudo[66313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:05:34 smithi188 sudo[66313]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:05:34 smithi188 sudo[66313]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:35.208 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:34 smithi110 sudo[85202]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:35.208 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:34 smithi110 sudo[85202]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:35.208 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:34 smithi110 sudo[85202]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:35.209 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:34 smithi110 sudo[85202]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:35.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 sudo[85205]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:35.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 sudo[85205]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:35.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 sudo[85205]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:35.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 sudo[85205]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 sudo[66317]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 sudo[66317]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 sudo[66317]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 sudo[66317]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:36.088 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:36.088 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: pgmap v457: 97 pgs: 97 active+clean; 1012 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.0 MiB/s wr, 1.03k op/s 2024-04-25T00:05:36.088 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:36.089 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:36.089 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:36.089 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:36.089 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:36.089 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:36.089 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 sudo[66320]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 sudo[66320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 sudo[66320]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 sudo[66320]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: pgmap v457: 97 pgs: 97 active+clean; 1012 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.0 MiB/s wr, 1.03k op/s 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:36.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:36.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:36.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:36.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:36.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: pgmap v457: 97 pgs: 97 active+clean; 1012 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.0 MiB/s wr, 1.03k op/s 2024-04-25T00:05:36.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:36.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:36.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:36.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:36.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:36.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:36.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 sudo[65276]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:05:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 sudo[65276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 sudo[65276]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:36.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 sudo[65276]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 150.0 s, 732.2 tps, lat 137.088 ms stddev 81.389 2024-04-25T00:05:36.817 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 sudo[65279]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:05:36.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 sudo[65279]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:05:36.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 sudo[65279]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:05:36.818 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 sudo[65279]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:05:37.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:37.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:37.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:37.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:37.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:37.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:37.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 ceph-mon[32402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:37.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:36 smithi173 ceph-mon[32402]: pgmap v458: 97 pgs: 97 active+clean; 1020 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.3 MiB/s wr, 984 op/s 2024-04-25T00:05:37.169 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:36 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:37.170 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:36 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:37.170 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:36 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:37.170 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:37.170 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:37.170 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:37.170 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:36 smithi188 ceph-mon[33285]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:37.170 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:36 smithi188 ceph-mon[33285]: pgmap v458: 97 pgs: 97 active+clean; 1020 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.3 MiB/s wr, 984 op/s 2024-04-25T00:05:37.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:36 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:37.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:36 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:37.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:36 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:05:37.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:05:37.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:05:37.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:05:37.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:36 smithi110 ceph-mon[29187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:05:37.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:36 smithi110 ceph-mon[29187]: pgmap v458: 97 pgs: 97 active+clean; 1020 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.3 MiB/s wr, 984 op/s 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 -- 172.21.15.110:0/3597194683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d4102270 msgr2=0x7f44d4108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/3597194683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d4102270 0x7f44d4108b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009ca0 tx=0x7f44cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 -- 172.21.15.110:0/3597194683 shutdown_connections 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/3597194683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d4102270 0x7f44d4108b00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/3597194683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44d41018b0 0x7f44d4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/3597194683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d41006c0 0x7f44d4100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 -- 172.21.15.110:0/3597194683 >> 172.21.15.110:0/3597194683 conn(0x7f44d40fbc50 msgr2=0x7f44d40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 -- 172.21.15.110:0/3597194683 shutdown_connections 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 -- 172.21.15.110:0/3597194683 wait complete. 2024-04-25T00:05:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.223+0000 7f44d9ffb640 1 Processor -- start 2024-04-25T00:05:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d9ffb640 1 -- start start 2024-04-25T00:05:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d9ffb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d41006c0 0x7f44d41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d41006c0 0x7f44d41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d9ffb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44d41018b0 0x7f44d4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d41006c0 0x7f44d41978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44560/0 (socket says 172.21.15.110:44560) 2024-04-25T00:05:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d9ffb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d4102270 0x7f44d419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d37fe640 1 -- 172.21.15.110:0/241367674 learned_addr learned my addr 172.21.15.110:0/241367674 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410d2d0 con 0x7f44d41006c0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410d150 con 0x7f44d41018b0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410d450 con 0x7f44d4102270 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44d41018b0 0x7f44d4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.224+0000 7f44d3fff640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d4102270 0x7f44d419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d37fe640 1 -- 172.21.15.110:0/241367674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d4102270 msgr2=0x7f44d419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d37fe640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d4102270 0x7f44d419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d37fe640 1 -- 172.21.15.110:0/241367674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44d41018b0 msgr2=0x7f44d4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d37fe640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44d41018b0 0x7f44d4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d37fe640 1 -- 172.21.15.110:0/241367674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d419f500 con 0x7f44d41006c0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d37fe640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d41006c0 0x7f44d41978e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009f20 tx=0x7f44c400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d0ff9640 1 -- 172.21.15.110:0/241367674 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c400cd00 con 0x7f44d41006c0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d0ff9640 1 -- 172.21.15.110:0/241367674 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c40137d0 con 0x7f44d41006c0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d0ff9640 1 -- 172.21.15.110:0/241367674 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4003850 con 0x7f44d41006c0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d41093a0 con 0x7f44d41006c0 2024-04-25T00:05:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.225+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d41097e0 con 0x7f44d41006c0 2024-04-25T00:05:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.229+0000 7f44d0ff9640 1 -- 172.21.15.110:0/241367674 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f44c4022020 con 0x7f44d41006c0 2024-04-25T00:05:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.229+0000 7f44dcde0740 1 Processor -- start 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.229+0000 7f44dcde0740 1 -- start start 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab4e50 0x5653c5ab5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab5840 0x5653c5aba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5abce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44dcde0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5653c5a18cb0 con 0x5653c5aba9b0 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44dcde0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5653c59c15b0 con 0x5653c5ab4e50 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44dcde0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5653c5a14c70 con 0x5653c5ab5840 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab5840 0x5653c5aba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5abce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab4e50 0x5653c5ab5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5abce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44574/0 (socket says 172.21.15.110:44574) 2024-04-25T00:05:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d2ffd640 1 -- 172.21.15.110:0/2373046728 learned_addr learned my addr 172.21.15.110:0/2373046728 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d0ff9640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44bc0a2cb0 0x7f44bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d2ffd640 1 -- 172.21.15.110:0/2373046728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab5840 msgr2=0x5653c5aba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/2373046728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab5840 0x5653c5aba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d2ffd640 1 -- 172.21.15.110:0/2373046728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab4e50 msgr2=0x5653c5ab5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/2373046728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab4e50 0x5653c5ab5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d2ffd640 1 -- 172.21.15.110:0/2373046728 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c572d210 con 0x5653c5aba9b0 2024-04-25T00:05:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d0ff9640 1 -- 172.21.15.110:0/241367674 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f44c40d3120 con 0x7f44d41006c0 2024-04-25T00:05:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.230+0000 7f44d3fff640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44bc0a2cb0 0x7f44bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.231+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/2373046728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5abce40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009c90 tx=0x7f44c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0a1299206e4e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.231+0000 7f44b9ffb640 1 -- 172.21.15.110:0/2373046728 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c801e070 con 0x5653c5aba9b0 2024-04-25T00:05:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.231+0000 7f44b9ffb640 1 -- 172.21.15.110:0/2373046728 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c8019ae0 con 0x5653c5aba9b0 2024-04-25T00:05:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.231+0000 7f44b9ffb640 1 -- 172.21.15.110:0/2373046728 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c800ea40 con 0x5653c5aba9b0 2024-04-25T00:05:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44d3fff640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44bc0a2cb0 0x7f44bc0a5170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f44cc005e30 tx=0x7f44cc0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 -- 172.21.15.110:0/2373046728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 msgr2=0x5653c5abce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2373046728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5abce40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009c90 tx=0x7f44c8011810 comp rx=0 tx=0).stop 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 -- 172.21.15.110:0/2373046728 shutdown_connections 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2373046728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5abce40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2373046728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab5840 0x5653c5aba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2373046728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab4e50 0x5653c5ab5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 -- 172.21.15.110:0/2373046728 >> 172.21.15.110:0/2373046728 conn(0x5653c5ab04c0 msgr2=0x5653c5ab08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 -- 172.21.15.110:0/2373046728 shutdown_connections 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.232+0000 7f44dcde0740 1 -- 172.21.15.110:0/2373046728 wait complete. 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 1 -- start start 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 1 Processor -- start 2024-04-25T00:05:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab4e50 0x5653c5abf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5abf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5ac67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5653c5abd960 con 0x5653c5aba9b0 2024-04-25T00:05:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5653c59d4b50 con 0x5653c5ab5840 2024-04-25T00:05:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5653c5abdae0 con 0x5653c5ab4e50 2024-04-25T00:05:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44dcde0740 10 client.0 _ll_register_callbacks cb 0x5653c5a0fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5abf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.233+0000 7f44d2ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5abf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:46504/0 (socket says 172.21.15.110:46504) 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d2ffd640 1 -- 172.21.15.110:0/2014889766 learned_addr learned my addr 172.21.15.110:0/2014889766 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d37fe640 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab4e50 0x5653c5abf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d3fff640 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5ac67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d2ffd640 1 -- 172.21.15.110:0/2014889766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab4e50 msgr2=0x5653c5abf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab4e50 0x5653c5abf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d2ffd640 1 -- 172.21.15.110:0/2014889766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 msgr2=0x5653c5ac67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5ac67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d2ffd640 1 -- 172.21.15.110:0/2014889766 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c57f9620 con 0x5653c5ab5840 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d3fff640 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5ac67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5abf790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44c800a010 tx=0x7f44c8019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b7a2dbc76b90bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:05:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44b2ffd640 1 -- 172.21.15.110:0/2014889766 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8029030 con 0x5653c5ab5840 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44b2ffd640 1 -- 172.21.15.110:0/2014889766 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f44c8019850 con 0x5653c5ab5840 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.234+0000 7f44dcde0740 1 -- 172.21.15.110:0/2014889766 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44c8019ae0 con 0x5653c5ab5840 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44b2ffd640 1 -- 172.21.15.110:0/2014889766 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44c801e040 con 0x5653c5ab5840 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44dcde0740 10 client.25130 fetch_fsmap learned FSMap version 28 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44dcde0740 1 -- 172.21.15.110:0/2014889766 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5653c5ac9b40 con 0x5653c5ab5840 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44b2ffd640 1 -- 172.21.15.110:0/2014889766 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c8019ae0 con 0x5653c5ab5840 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44b2ffd640 1 -- 172.21.15.110:0/2014889766 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f44c800ebc0 con 0x5653c5ab5840 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44dcde0740 10 client.25130 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44dcde0740 10 client.25130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44dcde0740 20 client.25130 populate_metadata read hostname 'smithi110' 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5653c5acb660 0x5653c5acdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44dcde0740 4 client.25130 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.235+0000 7f44dcde0740 1 -- 172.21.15.110:0/2014889766 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653c5a261f0 con 0x5653c5acb660 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.236+0000 7f44d37fe640 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5653c5acb660 0x5653c5acdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.236+0000 7f44b2ffd640 10 client.25130 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.236+0000 7f44d37fe640 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5653c5acb660 0x5653c5acdac0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424683c5bdd1722b server_cookie=14cd73162547dd59 in_seq=0 out_seq=0 2024-04-25T00:05:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44b2ffd640 1 -- 172.21.15.110:0/2014889766 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653c5a261f0 con 0x5653c5acb660 2024-04-25T00:05:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44b2ffd640 10 client.25130 handle_command_reply: tid=0 2024-04-25T00:05:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 client.25130 shutdown 2024-04-25T00:05:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 -- 172.21.15.110:0/2014889766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 msgr2=0x5653c5abf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5abf790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44c800a010 tx=0x7f44c8019660 comp rx=0 tx=0).stop 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 -- 172.21.15.110:0/2014889766 shutdown_connections 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5653c5acb660 0x5653c5acdac0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5aba9b0 0x5653c5ac67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5abf790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 --2- 172.21.15.110:0/2014889766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5ab4e50 0x5653c5abf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44b2ffd640 0 client.25130 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 -- 172.21.15.110:0/2014889766 >> 172.21.15.110:0/2014889766 conn(0x5653c5ab04c0 msgr2=0x5653c5aaf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 -- 172.21.15.110:0/2014889766 shutdown_connections 2024-04-25T00:05:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 1 -- 172.21.15.110:0/2014889766 wait complete. 2024-04-25T00:05:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.237+0000 7f44dcde0740 20 client.25130 trim_cache size 0 max 16384 2024-04-25T00:05:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.261+0000 7f44dcde0740 1 Processor -- start 2024-04-25T00:05:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.261+0000 7f44dcde0740 1 -- start start 2024-04-25T00:05:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.261+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ab6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ab6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 0x5653c5ab71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ab6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ab6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44596/0 (socket says 172.21.15.110:44596) 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 -- 172.21.15.110:0/3203670392 learned_addr learned my addr 172.21.15.110:0/3203670392 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ab6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44dcde0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a91b0 con 0x5653c5ab4e50 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44dcde0740 1 -- 172.21.15.110:0/3203670392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a69f0 con 0x5653c5ab5840 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44dcde0740 1 -- 172.21.15.110:0/3203670392 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a4840 con 0x5653c5aba9b0 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d3fff640 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 0x5653c5ab71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 -- 172.21.15.110:0/3203670392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 msgr2=0x5653c5ab71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 0x5653c5ab71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 -- 172.21.15.110:0/3203670392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 msgr2=0x5653c5ab6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ab6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 -- 172.21.15.110:0/3203670392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c5ab5c20 con 0x5653c5ab4e50 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ab6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.262+0000 7f44d37fe640 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ab6100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f44c4013fb0 tx=0x7f44c40d5570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7ed03eed3bcf69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.263+0000 7f44b97fa640 1 -- 172.21.15.110:0/3203670392 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c40a03a0 con 0x5653c5ab4e50 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.263+0000 7f44b97fa640 1 -- 172.21.15.110:0/3203670392 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c4013d10 con 0x5653c5ab4e50 2024-04-25T00:05:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.263+0000 7f44b97fa640 1 -- 172.21.15.110:0/3203670392 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c40a01c0 con 0x5653c5ab4e50 2024-04-25T00:05:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 -- 172.21.15.110:0/3203670392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 msgr2=0x5653c5ab6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ab6100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f44c4013fb0 tx=0x7f44c40d5570 comp rx=0 tx=0).stop 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 -- 172.21.15.110:0/3203670392 shutdown_connections 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 0x5653c5ab71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ab6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3203670392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ab6100 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 -- 172.21.15.110:0/3203670392 >> 172.21.15.110:0/3203670392 conn(0x5653c5ab04c0 msgr2=0x5653c5ab08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 -- 172.21.15.110:0/3203670392 shutdown_connections 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 -- 172.21.15.110:0/3203670392 wait complete. 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.264+0000 7f44dcde0740 1 -- start start 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44dcde0740 1 Processor -- start 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ad2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ad2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44dcde0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 0x5653c5b38470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44dcde0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a4eb0 con 0x5653c5ab4e50 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44dcde0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a6320 con 0x5653c5ab5840 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44dcde0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a5580 con 0x5653c5aba9b0 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44dcde0740 10 client.0 _ll_register_callbacks cb 0x7f44c801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ad2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44d37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ad2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44606/0 (socket says 172.21.15.110:44606) 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44d37fe640 1 -- 172.21.15.110:0/3048929787 learned_addr learned my addr 172.21.15.110:0/3048929787 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ad2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.265+0000 7f44d3fff640 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 0x5653c5b38470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44d2ffd640 1 -- 172.21.15.110:0/3048929787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 msgr2=0x5653c5b38470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 0x5653c5b38470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44d2ffd640 1 -- 172.21.15.110:0/3048929787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 msgr2=0x5653c5ad2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ad2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44d2ffd640 1 -- 172.21.15.110:0/3048929787 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c5b38a60 con 0x5653c5ab5840 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44d37fe640 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ad2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44d2ffd640 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ad2eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009820 tx=0x7f44c8019900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a70f64972af2a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44b2ffd640 1 -- 172.21.15.110:0/3048929787 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8029070 con 0x5653c5ab5840 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44b2ffd640 1 -- 172.21.15.110:0/3048929787 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f44c8004070 con 0x5653c5ab5840 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44dcde0740 1 -- 172.21.15.110:0/3048929787 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653c5a20540 con 0x5653c5ab5840 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44b2ffd640 1 -- 172.21.15.110:0/3048929787 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44c8004270 con 0x5653c5ab5840 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44dcde0740 10 client.25136 fetch_fsmap learned FSMap version 28 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.266+0000 7f44dcde0740 1 -- 172.21.15.110:0/3048929787 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5653c5b39d70 con 0x5653c5ab5840 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44b2ffd640 1 -- 172.21.15.110:0/3048929787 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5653c5a20540 con 0x5653c5ab5840 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44b2ffd640 1 -- 172.21.15.110:0/3048929787 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f44c8004de0 con 0x5653c5ab5840 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44dcde0740 10 client.25136 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:05:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44dcde0740 10 client.25136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:05:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44dcde0740 20 client.25136 populate_metadata read hostname 'smithi110' 2024-04-25T00:05:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5653c5b3a6a0 0x5653c5b3cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44dcde0740 4 client.25136 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:05:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44dcde0740 1 -- 172.21.15.110:0/3048929787 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653c599e8e0 con 0x5653c5b3a6a0 2024-04-25T00:05:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44d37fe640 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5653c5b3a6a0 0x5653c5b3cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.267+0000 7f44b2ffd640 10 client.25136 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.268+0000 7f44d37fe640 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5653c5b3a6a0 0x5653c5b3cac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5b32ba11539135 server_cookie=30471c33d7fd77da in_seq=0 out_seq=0 2024-04-25T00:05:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.285+0000 7f44b2ffd640 1 -- 172.21.15.110:0/3048929787 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653c599e8e0 con 0x5653c5b3a6a0 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.285+0000 7f44b2ffd640 10 client.25136 handle_command_reply: tid=0 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.285+0000 7f44dcde0740 1 client.25136 shutdown 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 -- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 msgr2=0x5653c5ad2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ad2eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009820 tx=0x7f44c8019900 comp rx=0 tx=0).stop 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 -- 172.21.15.110:0/3048929787 shutdown_connections 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5653c5b3a6a0 0x5653c5b3cac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653c5aba9b0 0x5653c5b38470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44b2ffd640 0 client.25136 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653c5ab5840 0x5653c5ad2eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 --2- 172.21.15.110:0/3048929787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5653c5ab4e50 0x5653c5ad2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 -- 172.21.15.110:0/3048929787 >> 172.21.15.110:0/3048929787 conn(0x5653c5ab04c0 msgr2=0x5653c5ab6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 -- 172.21.15.110:0/3048929787 shutdown_connections 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 1 -- 172.21.15.110:0/3048929787 wait complete. 2024-04-25T00:05:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.286+0000 7f44dcde0740 20 client.25136 trim_cache size 0 max 16384 2024-04-25T00:05:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:05:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:05:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:05:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:05:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44bc0a2cb0 msgr2=0x7f44bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44bc0a2cb0 0x7f44bc0a5170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f44cc005e30 tx=0x7f44cc0095c0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d41006c0 msgr2=0x7f44d41978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d41006c0 0x7f44d41978e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009f20 tx=0x7f44c400ee50 comp rx=0 tx=0).stop 2024-04-25T00:05:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 shutdown_connections 2024-04-25T00:05:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44bc0a2cb0 0x7f44bc0a5170 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d4102270 0x7f44d419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44d41018b0 0x7f44d4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 --2- 172.21.15.110:0/241367674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44d41006c0 0x7f44d41978e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 >> 172.21.15.110:0/241367674 conn(0x7f44d40fbc50 msgr2=0x7f44d4104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 shutdown_connections 2024-04-25T00:05:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:38.287+0000 7f44d9ffb640 1 -- 172.21.15.110:0/241367674 wait complete. 2024-04-25T00:05:40.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:39 smithi188 ceph-mon[33285]: pgmap v459: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-04-25T00:05:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:39 smithi110 ceph-mon[29187]: pgmap v459: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-04-25T00:05:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:39 smithi173 ceph-mon[32402]: pgmap v459: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-04-25T00:05:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 155.0 s, 806.8 tps, lat 124.123 ms stddev 15.477 2024-04-25T00:05:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:41 smithi188 ceph-mon[33285]: pgmap v460: 97 pgs: 97 active+clean; 1021 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.5 MiB/s wr, 980 op/s 2024-04-25T00:05:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:41 smithi110 ceph-mon[29187]: pgmap v460: 97 pgs: 97 active+clean; 1021 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.5 MiB/s wr, 980 op/s 2024-04-25T00:05:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:41 smithi173 ceph-mon[32402]: pgmap v460: 97 pgs: 97 active+clean; 1021 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.5 MiB/s wr, 980 op/s 2024-04-25T00:05:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:43 smithi188 ceph-mon[33285]: pgmap v461: 97 pgs: 97 active+clean; 1021 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:05:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:43 smithi110 ceph-mon[29187]: pgmap v461: 97 pgs: 97 active+clean; 1021 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:05:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:43 smithi173 ceph-mon[32402]: pgmap v461: 97 pgs: 97 active+clean; 1021 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:05:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:45 smithi188 ceph-mon[33285]: pgmap v462: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.9 MiB/s wr, 930 op/s 2024-04-25T00:05:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:45 smithi110 ceph-mon[29187]: pgmap v462: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.9 MiB/s wr, 930 op/s 2024-04-25T00:05:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:45 smithi173 ceph-mon[32402]: pgmap v462: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.9 MiB/s wr, 930 op/s 2024-04-25T00:05:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 160.0 s, 822.6 tps, lat 121.503 ms stddev 11.657 2024-04-25T00:05:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:47 smithi188 ceph-mon[33285]: pgmap v463: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 963 op/s 2024-04-25T00:05:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:47 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:47 smithi110 ceph-mon[29187]: pgmap v463: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 963 op/s 2024-04-25T00:05:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:47 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:47 smithi173 ceph-mon[32402]: pgmap v463: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 963 op/s 2024-04-25T00:05:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:47 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:50.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:49 smithi188 ceph-mon[33285]: pgmap v464: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-04-25T00:05:50.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:49 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:49 smithi110 ceph-mon[29187]: pgmap v464: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-04-25T00:05:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:49 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:49 smithi173 ceph-mon[32402]: pgmap v464: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-04-25T00:05:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:49 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:05:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.421+0000 7f5ac84df640 1 -- 172.21.15.110:0/3754433322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac0102280 msgr2=0x7f5ac0104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.421+0000 7f5ac84df640 1 --2- 172.21.15.110:0/3754433322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac0102280 0x7f5ac0104670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009900 tx=0x7f5abc02a410 comp rx=0 tx=0).stop 2024-04-25T00:05:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 -- 172.21.15.110:0/3754433322 shutdown_connections 2024-04-25T00:05:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 --2- 172.21.15.110:0/3754433322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac01074e0 0x7f5ac01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 --2- 172.21.15.110:0/3754433322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ac0104bb0 0x7f5ac0106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 --2- 172.21.15.110:0/3754433322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac0102280 0x7f5ac0104670 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 -- 172.21.15.110:0/3754433322 >> 172.21.15.110:0/3754433322 conn(0x7f5ac00fbc10 msgr2=0x7f5ac00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 -- 172.21.15.110:0/3754433322 shutdown_connections 2024-04-25T00:05:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 -- 172.21.15.110:0/3754433322 wait complete. 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 Processor -- start 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.422+0000 7f5ac84df640 1 -- start start 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac84df640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac0102280 0x7f5ac0101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac84df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac0104bb0 0x7f5ac0100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac84df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ac01074e0 0x7f5ac0100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac84df640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac010d310 con 0x7f5ac0104bb0 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac84df640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac010d190 con 0x7f5ac0102280 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac84df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac010d490 con 0x7f5ac01074e0 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac0102280 0x7f5ac0101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac0102280 0x7f5ac0101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43200/0 (socket says 172.21.15.110:43200) 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 -- 172.21.15.110:0/1326973941 learned_addr learned my addr 172.21.15.110:0/1326973941 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ac01074e0 0x7f5ac0100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac0104bb0 0x7f5ac0100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 -- 172.21.15.110:0/1326973941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ac01074e0 msgr2=0x7f5ac0100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ac01074e0 0x7f5ac0100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 -- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac0104bb0 msgr2=0x7f5ac0100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac0104bb0 0x7f5ac0100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 -- 172.21.15.110:0/1326973941 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac0101000 con 0x7f5ac0102280 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.423+0000 7f5ac627b640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac0102280 0x7f5ac0101bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0314e0 tx=0x7f5abc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.424+0000 7f5ab77fe640 1 -- 172.21.15.110:0/1326973941 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc03f070 con 0x7f5ac0102280 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.424+0000 7f5ac84df640 1 -- 172.21.15.110:0/1326973941 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac01a1a40 con 0x7f5ac0102280 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.424+0000 7f5ab77fe640 1 -- 172.21.15.110:0/1326973941 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5abc031c40 con 0x7f5ac0102280 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.424+0000 7f5ac84df640 1 -- 172.21.15.110:0/1326973941 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac01a1fe0 con 0x7f5ac0102280 2024-04-25T00:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.424+0000 7f5ab77fe640 1 -- 172.21.15.110:0/1326973941 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc035030 con 0x7f5ac0102280 2024-04-25T00:05:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.427+0000 7f5ab77fe640 1 -- 172.21.15.110:0/1326973941 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5abc042070 con 0x7f5ac0102280 2024-04-25T00:05:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.427+0000 7f5acb1df740 1 Processor -- start 2024-04-25T00:05:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.427+0000 7f5acb1df740 1 -- start start 2024-04-25T00:05:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.427+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ee6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.427+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65eebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.427+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65eec1c0 0x55cb65eee650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.427+0000 7f5acb1df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cb65e41cb0 con 0x55cb65ee7050 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5acb1df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cb65dea5b0 con 0x55cb65eec1c0 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5acb1df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cb65e3dc70 con 0x55cb65ee6660 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac627b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65eebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac6a7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ee6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac627b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65eebb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44266/0 (socket says 172.21.15.110:44266) 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac627b640 1 -- 172.21.15.110:0/1076458050 learned_addr learned my addr 172.21.15.110:0/1076458050 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65eec1c0 0x55cb65eee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/1076458050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65eec1c0 msgr2=0x55cb65eee650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65eec1c0 0x55cb65eee650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/1076458050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 msgr2=0x55cb65eebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65eebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/1076458050 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb65b56210 con 0x55cb65ee6660 2024-04-25T00:05:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ac627b640 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65eebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:05:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.428+0000 7f5ab77fe640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5aac0a2d30 0x7f5aac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.429+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ee6a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009f00 tx=0x7f5ab8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa15892c2f8370e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.429+0000 7f5ac627b640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5aac0a2d30 0x7f5aac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.429+0000 7f5ab4bfe640 1 -- 172.21.15.110:0/1076458050 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8004d20 con 0x55cb65ee6660 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.429+0000 7f5ab4bfe640 1 -- 172.21.15.110:0/1076458050 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ab8004030 con 0x55cb65ee6660 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.429+0000 7f5ab4bfe640 1 -- 172.21.15.110:0/1076458050 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8005280 con 0x55cb65ee6660 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.429+0000 7f5ab77fe640 1 -- 172.21.15.110:0/1326973941 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5abc0ef5d0 con 0x7f5ac0102280 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 -- 172.21.15.110:0/1076458050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 msgr2=0x55cb65ee6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5ac627b640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5aac0a2d30 0x7f5aac0a51f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5abc03b910 tx=0x7f5abc0f2f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ee6a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009f00 tx=0x7f5ab8011810 comp rx=0 tx=0).stop 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 -- 172.21.15.110:0/1076458050 shutdown_connections 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65eec1c0 0x55cb65eee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65eebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1076458050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ee6a80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 -- 172.21.15.110:0/1076458050 >> 172.21.15.110:0/1076458050 conn(0x55cb65ed94c0 msgr2=0x55cb65ed98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 -- 172.21.15.110:0/1076458050 shutdown_connections 2024-04-25T00:05:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 -- 172.21.15.110:0/1076458050 wait complete. 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 -- start start 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.430+0000 7f5acb1df740 1 Processor -- start 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee6660 0x55cb65ef0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65ef1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 0x55cb65ef8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5acb1df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cb65eef170 con 0x55cb65ee7050 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5acb1df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cb65dfdb50 con 0x55cb65ee6660 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5acb1df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cb65eef2f0 con 0x55cb65eec1c0 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5acb1df740 10 client.0 _ll_register_callbacks cb 0x55cb65e38ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 0x55cb65ef8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee6660 0x55cb65ef0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee6660 0x55cb65ef0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43212/0 (socket says 172.21.15.110:43212) 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/1931101679 learned_addr learned my addr 172.21.15.110:0/1931101679 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65ef1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/1931101679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 msgr2=0x55cb65ef8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 0x55cb65ef8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/1931101679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 msgr2=0x55cb65ef1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65ef1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/1931101679 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb65c22620 con 0x55cb65ee6660 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65ef1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee6660 0x55cb65ef0ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80046e0 tx=0x7f5ab8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87d2142dc33b0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.431+0000 7f5aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.432+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/1931101679 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab802d060 con 0x55cb65ee6660 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.432+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/1931101679 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5ab8004c40 con 0x55cb65ee6660 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.432+0000 7f5acb1df740 1 -- 172.21.15.110:0/1931101679 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab8004030 con 0x55cb65ee6660 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.432+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/1931101679 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ab801e040 con 0x55cb65ee6660 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.432+0000 7f5acb1df740 10 client.25148 fetch_fsmap learned FSMap version 30 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.432+0000 7f5acb1df740 1 -- 172.21.15.110:0/1931101679 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb65efb3f0 con 0x55cb65ee6660 2024-04-25T00:05:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.432+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/1931101679 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ab8004030 con 0x55cb65ee6660 2024-04-25T00:05:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.432+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/1931101679 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5ab8024070 con 0x55cb65ee6660 2024-04-25T00:05:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5acb1df740 10 client.25148 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:05:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5acb1df740 10 client.25148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:05:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5acb1df740 20 client.25148 populate_metadata read hostname 'smithi110' 2024-04-25T00:05:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55cb65efc910 0x55cb65efed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5acb1df740 4 client.25148 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:05:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5acb1df740 1 -- 172.21.15.110:0/1931101679 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb65e4f1f0 con 0x55cb65efc910 2024-04-25T00:05:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55cb65efc910 0x55cb65efed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55cb65efc910 0x55cb65efed30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a70aa4c6be4b3e server_cookie=2d23bd545637a987 in_seq=0 out_seq=0 2024-04-25T00:05:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.433+0000 7f5aa9ffb640 10 client.25148 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.434+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/1931101679 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb65e4f1f0 con 0x55cb65efc910 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.434+0000 7f5aa9ffb640 10 client.25148 handle_command_reply: tid=0 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.434+0000 7f5acb1df740 1 client.25148 shutdown 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.434+0000 7f5acb1df740 1 -- 172.21.15.110:0/1931101679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee6660 msgr2=0x55cb65ef0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.434+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee6660 0x55cb65ef0ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80046e0 tx=0x7f5ab8000f40 comp rx=0 tx=0).stop 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 1 -- 172.21.15.110:0/1931101679 shutdown_connections 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55cb65efc910 0x55cb65efed30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 0x55cb65ef8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee7050 0x55cb65ef1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1931101679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee6660 0x55cb65ef0ad0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 1 -- 172.21.15.110:0/1931101679 >> 172.21.15.110:0/1931101679 conn(0x55cb65ed94c0 msgr2=0x55cb65ed85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5aa9ffb640 0 client.25148 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 1 -- 172.21.15.110:0/1931101679 shutdown_connections 2024-04-25T00:05:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 1 -- 172.21.15.110:0/1931101679 wait complete. 2024-04-25T00:05:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.435+0000 7f5acb1df740 20 client.25148 trim_cache size 0 max 16384 2024-04-25T00:05:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.457+0000 7f5acb1df740 1 Processor -- start 2024-04-25T00:05:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5acb1df740 1 -- start start 2024-04-25T00:05:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ed7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65eec1c0 0x55cb65ed8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5acb1df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa4010cd0 con 0x55cb65eec1c0 2024-04-25T00:05:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5acb1df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa400e540 con 0x55cb65ee7050 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5acb1df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa400c390 con 0x55cb65ee6660 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac5a7a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac5a7a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43226/0 (socket says 172.21.15.110:43226) 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac5a7a640 1 -- 172.21.15.110:0/1661486894 learned_addr learned my addr 172.21.15.110:0/1661486894 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ed7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65eec1c0 0x55cb65ed8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac5a7a640 1 -- 172.21.15.110:0/1661486894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 msgr2=0x55cb65ed7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ed7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac5a7a640 1 -- 172.21.15.110:0/1661486894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65eec1c0 msgr2=0x55cb65ed8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65eec1c0 0x55cb65ed8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.458+0000 7f5ac5a7a640 1 -- 172.21.15.110:0/1661486894 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb65eeb730 con 0x55cb65ee7050 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.459+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef8010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55cb65eda430 tx=0x7f5ab0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a6630e56c90f62a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.459+0000 7f5aabfff640 1 -- 172.21.15.110:0/1661486894 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0024070 con 0x55cb65ee7050 2024-04-25T00:05:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.459+0000 7f5aabfff640 1 -- 172.21.15.110:0/1661486894 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ab001e930 con 0x55cb65ee7050 2024-04-25T00:05:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.459+0000 7f5aabfff640 1 -- 172.21.15.110:0/1661486894 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0003ca0 con 0x55cb65ee7050 2024-04-25T00:05:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 -- 172.21.15.110:0/1661486894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 msgr2=0x55cb65ef8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef8010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55cb65eda430 tx=0x7f5ab0005f90 comp rx=0 tx=0).stop 2024-04-25T00:05:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 -- 172.21.15.110:0/1661486894 shutdown_connections 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65eec1c0 0x55cb65ed8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef8010 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 --2- 172.21.15.110:0/1661486894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65ee6660 0x55cb65ed7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 -- 172.21.15.110:0/1661486894 >> 172.21.15.110:0/1661486894 conn(0x55cb65ed94c0 msgr2=0x55cb65ef06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 -- 172.21.15.110:0/1661486894 shutdown_connections 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 -- 172.21.15.110:0/1661486894 wait complete. 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.460+0000 7f5acb1df740 1 -- start start 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5acb1df740 1 Processor -- start 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee6660 0x55cb65ef59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5acb1df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 0x55cb65ef6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5acb1df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa400ca00 con 0x55cb65ee6660 2024-04-25T00:05:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5acb1df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa400de70 con 0x55cb65ee7050 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5acb1df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa400d0d0 con 0x55cb65eec1c0 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee6660 0x55cb65ef59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5acb1df740 10 client.0 _ll_register_callbacks cb 0x7f5abc0f2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee6660 0x55cb65ef59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44292/0 (socket says 172.21.15.110:44292) 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/612506391 learned_addr learned my addr 172.21.15.110:0/612506391 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 0x55cb65ef6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/612506391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 msgr2=0x55cb65ef6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 0x55cb65ef6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/612506391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 msgr2=0x55cb65ef5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.461+0000 7f5ac6a7c640 1 -- 172.21.15.110:0/612506391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb65ef7510 con 0x55cb65ee6660 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5ac6a7c640 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee6660 0x55cb65ef59d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8003e10 tx=0x7f5ab8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1643ed3732065109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:05:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/612506391 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab802d070 con 0x55cb65ee6660 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/612506391 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5ab8005b90 con 0x55cb65ee6660 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5acb1df740 1 -- 172.21.15.110:0/612506391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab001e930 con 0x55cb65ee6660 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/612506391 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ab8034a80 con 0x55cb65ee6660 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5acb1df740 10 client.15573 fetch_fsmap learned FSMap version 30 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5acb1df740 1 -- 172.21.15.110:0/612506391 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb65f63150 con 0x55cb65ee6660 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.462+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/612506391 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ab803b040 con 0x55cb65ee6660 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/612506391 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5ab8040070 con 0x55cb65ee6660 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5acb1df740 10 client.15573 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5acb1df740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5acb1df740 20 client.15573 populate_metadata read hostname 'smithi110' 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5acb1df740 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55cb65f63b80 0x55cb65f65fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:05:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5acb1df740 4 client.15573 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:05:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5acb1df740 1 -- 172.21.15.110:0/612506391 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5ab001e930 con 0x55cb65f63b80 2024-04-25T00:05:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55cb65f63b80 0x55cb65f65fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:05:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.463+0000 7f5aa9ffb640 10 client.15573 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.464+0000 7f5ac5a7a640 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55cb65f63b80 0x55cb65f65fa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9093ea897e1af6 server_cookie=83e848e17e72c888 in_seq=0 out_seq=0 2024-04-25T00:05:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5aa9ffb640 1 -- 172.21.15.110:0/612506391 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5ab001e930 con 0x55cb65f63b80 2024-04-25T00:05:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5aa9ffb640 10 client.15573 handle_command_reply: tid=0 2024-04-25T00:05:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 client.15573 shutdown 2024-04-25T00:05:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 -- 172.21.15.110:0/612506391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee6660 msgr2=0x55cb65ef59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee6660 0x55cb65ef59d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8003e10 tx=0x7f5ab8014040 comp rx=0 tx=0).stop 2024-04-25T00:05:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 -- 172.21.15.110:0/612506391 shutdown_connections 2024-04-25T00:05:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55cb65f63b80 0x55cb65f65fa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb65eec1c0 0x55cb65ef6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb65ee7050 0x55cb65ef5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 --2- 172.21.15.110:0/612506391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cb65ee6660 0x55cb65ef59d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5aa9ffb640 0 client.15573 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.479+0000 7f5acb1df740 1 -- 172.21.15.110:0/612506391 >> 172.21.15.110:0/612506391 conn(0x55cb65ed94c0 msgr2=0x55cb65ed7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.480+0000 7f5acb1df740 1 -- 172.21.15.110:0/612506391 shutdown_connections 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.480+0000 7f5acb1df740 1 -- 172.21.15.110:0/612506391 wait complete. 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.480+0000 7f5acb1df740 20 client.15573 trim_cache size 0 max 16384 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:05:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:05:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.480+0000 7f5ac84df640 1 -- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5aac0a2d30 msgr2=0x7f5aac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.480+0000 7f5ac84df640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5aac0a2d30 0x7f5aac0a51f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5abc03b910 tx=0x7f5abc0f2f90 comp rx=0 tx=0).stop 2024-04-25T00:05:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.480+0000 7f5ac84df640 1 -- 172.21.15.110:0/1326973941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac0102280 msgr2=0x7f5ac0101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:05:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.480+0000 7f5ac84df640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac0102280 0x7f5ac0101bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0314e0 tx=0x7f5abc003c10 comp rx=0 tx=0).stop 2024-04-25T00:05:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.481+0000 7f5ac84df640 1 -- 172.21.15.110:0/1326973941 shutdown_connections 2024-04-25T00:05:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.481+0000 7f5ac84df640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5aac0a2d30 0x7f5aac0a51f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.481+0000 7f5ac84df640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ac01074e0 0x7f5ac0100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.481+0000 7f5ac84df640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac0104bb0 0x7f5ac0100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.481+0000 7f5ac84df640 1 --2- 172.21.15.110:0/1326973941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac0102280 0x7f5ac0101bf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:05:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.481+0000 7f5ac84df640 1 -- 172.21.15.110:0/1326973941 >> 172.21.15.110:0/1326973941 conn(0x7f5ac00fbc10 msgr2=0x7f5ac0102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:05:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.481+0000 7f5ac84df640 1 -- 172.21.15.110:0/1326973941 shutdown_connections 2024-04-25T00:05:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:05:51.481+0000 7f5ac84df640 1 -- 172.21.15.110:0/1326973941 wait complete. 2024-04-25T00:05:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 165.0 s, 808.6 tps, lat 123.543 ms stddev 15.567 2024-04-25T00:05:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:51 smithi188 ceph-mon[33285]: pgmap v465: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-04-25T00:05:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:51 smithi110 ceph-mon[29187]: pgmap v465: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-04-25T00:05:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:51 smithi173 ceph-mon[32402]: pgmap v465: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-04-25T00:05:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:53 smithi188 ceph-mon[33285]: pgmap v466: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 992 op/s 2024-04-25T00:05:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:53 smithi110 ceph-mon[29187]: pgmap v466: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 992 op/s 2024-04-25T00:05:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:53 smithi173 ceph-mon[32402]: pgmap v466: 97 pgs: 97 active+clean; 1021 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 992 op/s 2024-04-25T00:05:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:55 smithi188 ceph-mon[33285]: pgmap v467: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 1.13k op/s 2024-04-25T00:05:56.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:55 smithi110 ceph-mon[29187]: pgmap v467: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 1.13k op/s 2024-04-25T00:05:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:55 smithi173 ceph-mon[32402]: pgmap v467: 97 pgs: 97 active+clean; 1021 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 1.13k op/s 2024-04-25T00:05:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 170.0 s, 854.4 tps, lat 117.085 ms stddev 12.904 2024-04-25T00:05:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:57 smithi188 ceph-mon[33285]: pgmap v468: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 1.05k op/s 2024-04-25T00:05:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:57 smithi110 ceph-mon[29187]: pgmap v468: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 1.05k op/s 2024-04-25T00:05:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:57 smithi173 ceph-mon[32402]: pgmap v468: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 1.05k op/s 2024-04-25T00:06:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:05:59 smithi188 ceph-mon[33285]: pgmap v469: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-04-25T00:06:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:05:59 smithi110 ceph-mon[29187]: pgmap v469: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-04-25T00:06:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:05:59 smithi173 ceph-mon[32402]: pgmap v469: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-04-25T00:06:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 175.0 s, 802.0 tps, lat 124.563 ms stddev 12.235 2024-04-25T00:06:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:01 smithi188 ceph-mon[33285]: pgmap v470: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 1.05k op/s 2024-04-25T00:06:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:01 smithi110 ceph-mon[29187]: pgmap v470: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 1.05k op/s 2024-04-25T00:06:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:01 smithi173 ceph-mon[32402]: pgmap v470: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 1.05k op/s 2024-04-25T00:06:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:02 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:06:03.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:02 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:06:03.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:02 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:06:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:03 smithi188 ceph-mon[33285]: pgmap v471: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:06:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:03 smithi110 ceph-mon[29187]: pgmap v471: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:06:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:03 smithi173 ceph-mon[32402]: pgmap v471: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 1.12k op/s 2024-04-25T00:06:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 -- 172.21.15.110:0/3720153614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410dd40 msgr2=0x7f74241145d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 --2- 172.21.15.110:0/3720153614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410dd40 0x7f74241145d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f741c009ca0 tx=0x7f741c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 -- 172.21.15.110:0/3720153614 shutdown_connections 2024-04-25T00:06:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 --2- 172.21.15.110:0/3720153614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410dd40 0x7f74241145d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 --2- 172.21.15.110:0/3720153614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f742410d380 0x7f742410d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 --2- 172.21.15.110:0/3720153614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742410c190 0x7f742410c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 -- 172.21.15.110:0/3720153614 >> 172.21.15.110:0/3720153614 conn(0x7f742406c360 msgr2=0x7f742406c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 -- 172.21.15.110:0/3720153614 shutdown_connections 2024-04-25T00:06:04.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.608+0000 7f7428cca640 1 -- 172.21.15.110:0/3720153614 wait complete. 2024-04-25T00:06:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7428cca640 1 Processor -- start 2024-04-25T00:06:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7428cca640 1 -- start start 2024-04-25T00:06:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7428cca640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410c190 0x7f74241a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7428cca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f742410d380 0x7f74241a2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7428cca640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742410dd40 0x7f74241a9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7428cca640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7424117cd0 con 0x7f742410d380 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7428cca640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7424117b50 con 0x7f742410dd40 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7428cca640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7424117e50 con 0x7f742410c190 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f742259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410c190 0x7f74241a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f742259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410c190 0x7f74241a22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:45706/0 (socket says 172.21.15.110:45706) 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f742259c640 1 -- 172.21.15.110:0/1787243305 learned_addr learned my addr 172.21.15.110:0/1787243305 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f7422d9d640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742410dd40 0x7f74241a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f741bfff640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f742410d380 0x7f74241a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f742259c640 1 -- 172.21.15.110:0/1787243305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742410dd40 msgr2=0x7f74241a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f742259c640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742410dd40 0x7f74241a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f742259c640 1 -- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f742410d380 msgr2=0x7f74241a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f742259c640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f742410d380 0x7f74241a2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.609+0000 7f742259c640 1 -- 172.21.15.110:0/1787243305 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74241a9ef0 con 0x7f742410c190 2024-04-25T00:06:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.610+0000 7f741bfff640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f742410d380 0x7f74241a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.610+0000 7f742259c640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410c190 0x7f74241a22d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f741400d950 tx=0x7f741400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.610+0000 7f741b7fe640 1 -- 172.21.15.110:0/1787243305 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414017020 con 0x7f742410c190 2024-04-25T00:06:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.610+0000 7f741b7fe640 1 -- 172.21.15.110:0/1787243305 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7414009c30 con 0x7f742410c190 2024-04-25T00:06:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.610+0000 7f7428cca640 1 -- 172.21.15.110:0/1787243305 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74241aa1e0 con 0x7f742410c190 2024-04-25T00:06:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.610+0000 7f741b7fe640 1 -- 172.21.15.110:0/1787243305 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741400f6b0 con 0x7f742410c190 2024-04-25T00:06:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.610+0000 7f7428cca640 1 -- 172.21.15.110:0/1787243305 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74241aa730 con 0x7f742410c190 2024-04-25T00:06:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.613+0000 7f741b7fe640 1 -- 172.21.15.110:0/1787243305 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f741401a020 con 0x7f742410c190 2024-04-25T00:06:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.613+0000 7f742b9d7740 1 Processor -- start 2024-04-25T00:06:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742b9d7740 1 -- start start 2024-04-25T00:06:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d49e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 0x55e71d4a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742b9d7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e71d401cb0 con 0x55e71d49e840 2024-04-25T00:06:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742b9d7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e71d3aa5b0 con 0x55e71d4a39b0 2024-04-25T00:06:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742b9d7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e71d3fdc70 con 0x55e71d49de50 2024-04-25T00:06:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f7422d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d49e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49066/0 (socket says 172.21.15.110:49066) 2024-04-25T00:06:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f742259c640 1 -- 172.21.15.110:0/3164969184 learned_addr learned my addr 172.21.15.110:0/3164969184 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.614+0000 7f741bfff640 1 --2- 172.21.15.110:0/3164969184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 0x55e71d4a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7422d9d640 1 -- 172.21.15.110:0/3164969184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 msgr2=0x55e71d4a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f741b7fe640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f74100a2e20 0x7f74100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7422d9d640 1 --2- 172.21.15.110:0/3164969184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 0x55e71d4a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7422d9d640 1 -- 172.21.15.110:0/3164969184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 msgr2=0x55e71d4a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7422d9d640 1 --2- 172.21.15.110:0/3164969184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7422d9d640 1 -- 172.21.15.110:0/3164969184 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71d116210 con 0x55e71d49de50 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7422d9d640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f74100a2e20 0x7f74100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f741b7fe640 1 -- 172.21.15.110:0/1787243305 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f74140cef00 con 0x7f742410c190 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7422d9d640 1 --2- 172.21.15.110:0/3164969184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d49e270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f741c031040 tx=0x7f741c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d76dc96f73ec12d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7418ff9640 1 -- 172.21.15.110:0/3164969184 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c031b40 con 0x55e71d49de50 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7418ff9640 1 -- 172.21.15.110:0/3164969184 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741c036030 con 0x55e71d49de50 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.615+0000 7f7418ff9640 1 -- 172.21.15.110:0/3164969184 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c0305c0 con 0x55e71d49de50 2024-04-25T00:06:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.616+0000 7f742b9d7740 1 -- 172.21.15.110:0/3164969184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 msgr2=0x55e71d49e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.616+0000 7f742b9d7740 1 --2- 172.21.15.110:0/3164969184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d49e270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f741c031040 tx=0x7f741c009a20 comp rx=0 tx=0).stop 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.616+0000 7f7422d9d640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f74100a2e20 0x7f74100a52e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55e71d4aee40 tx=0x7f741c0308a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.616+0000 7f742b9d7740 1 -- 172.21.15.110:0/3164969184 shutdown_connections 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.616+0000 7f742b9d7740 1 --2- 172.21.15.110:0/3164969184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 0x55e71d4a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.616+0000 7f742b9d7740 1 --2- 172.21.15.110:0/3164969184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.616+0000 7f742b9d7740 1 --2- 172.21.15.110:0/3164969184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d49e270 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.616+0000 7f742b9d7740 1 -- 172.21.15.110:0/3164969184 >> 172.21.15.110:0/3164969184 conn(0x55e71d4994c0 msgr2=0x55e71d4998c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.617+0000 7f742b9d7740 1 -- 172.21.15.110:0/3164969184 shutdown_connections 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.617+0000 7f742b9d7740 1 -- 172.21.15.110:0/3164969184 wait complete. 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.617+0000 7f742b9d7740 1 -- start start 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.617+0000 7f742b9d7740 1 Processor -- start 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49072/0 (socket says 172.21.15.110:49072) 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742259c640 1 -- 172.21.15.110:0/1507561145 learned_addr learned my addr 172.21.15.110:0/1507561145 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742b9d7740 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 0x55e71d4b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e71d4aeff0 con 0x55e71d49de50 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e71d3bdb50 con 0x55e71d4a39b0 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f741bfff640 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e71d4af170 con 0x55e71d49e840 2024-04-25T00:06:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.618+0000 7f7422d9d640 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 0x55e71d4b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f741bfff640 1 -- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 msgr2=0x55e71d4b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f741bfff640 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 0x55e71d4b7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f741bfff640 1 -- 172.21.15.110:0/1507561145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 msgr2=0x55e71d4b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f741bfff640 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f741bfff640 1 -- 172.21.15.110:0/1507561145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71d1e2620 con 0x55e71d49e840 2024-04-25T00:06:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f742b9d7740 10 client.0 _ll_register_callbacks cb 0x55e71d3f8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f741c036030 con 0x55e71d49e840 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f741bfff640 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b0f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74241a37e0 tx=0x7f740c00dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd1b7a2c3978e588 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f7409ffb640 10 client.25147 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f7409ffb640 1 -- 172.21.15.110:0/1507561145 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c01b440 con 0x55e71d49e840 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f7409ffb640 1 -- 172.21.15.110:0/1507561145 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f740c01b5e0 con 0x55e71d49e840 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f7409ffb640 1 -- 172.21.15.110:0/1507561145 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f740c01b840 con 0x55e71d49e840 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f742b9d7740 10 client.25147 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.619+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55e71d4bb2b0 con 0x55e71d49e840 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.620+0000 7f7409ffb640 1 -- 172.21.15.110:0/1507561145 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f740c019070 con 0x55e71d49e840 2024-04-25T00:06:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.620+0000 7f742b9d7740 10 client.25147 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.620+0000 7f742b9d7740 10 client.25147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.620+0000 7f742b9d7740 20 client.25147 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.620+0000 7f742b9d7740 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e71d4bc960 0x55e71d4bedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.620+0000 7f742b9d7740 4 client.25147 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.620+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e71d40f1f0 con 0x55e71d4bc960 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.620+0000 7f742259c640 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e71d4bc960 0x55e71d4bedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.621+0000 7f742259c640 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e71d4bc960 0x55e71d4bedf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c27cc2c811443e server_cookie=803e24fd5ee67738 in_seq=0 out_seq=0 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.621+0000 7f7409ffb640 10 client.25147 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.621+0000 7f7409ffb640 1 -- 172.21.15.110:0/1507561145 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e71d40f1f0 con 0x55e71d4bc960 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.621+0000 7f7409ffb640 10 client.25147 handle_command_reply: tid=0 2024-04-25T00:06:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.621+0000 7f742b9d7740 1 client.25147 shutdown 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 msgr2=0x55e71d4b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b0f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74241a37e0 tx=0x7f740c00dd70 comp rx=0 tx=0).stop 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 shutdown_connections 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e71d4bc960 0x55e71d4bedf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4a39b0 0x55e71d4b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b0f00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f7409ffb640 0 client.25147 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 --2- 172.21.15.110:0/1507561145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 >> 172.21.15.110:0/1507561145 conn(0x55e71d4994c0 msgr2=0x55e71d4985c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 shutdown_connections 2024-04-25T00:06:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 1 -- 172.21.15.110:0/1507561145 wait complete. 2024-04-25T00:06:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.622+0000 7f742b9d7740 20 client.25147 trim_cache size 0 max 16384 2024-04-25T00:06:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.646+0000 7f742b9d7740 1 Processor -- start 2024-04-25T00:06:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.646+0000 7f742b9d7740 1 -- start start 2024-04-25T00:06:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d498440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d4b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f742b9d7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f74241b18d0 con 0x55e71d49e840 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f742b9d7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74241af170 con 0x55e71d4994c0 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f742b9d7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74241acfe0 con 0x55e71d49de50 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f741bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d4b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f7422d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f741bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d4b7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:45716/0 (socket says 172.21.15.110:45716) 2024-04-25T00:06:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f741bfff640 1 -- 172.21.15.110:0/4023458832 learned_addr learned my addr 172.21.15.110:0/4023458832 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f742259c640 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d498440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f7422d9d640 1 -- 172.21.15.110:0/4023458832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 msgr2=0x55e71d4b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f7422d9d640 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d4b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f7422d9d640 1 -- 172.21.15.110:0/4023458832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 msgr2=0x55e71d498440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f7422d9d640 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d498440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f7422d9d640 1 -- 172.21.15.110:0/4023458832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71d49d450 con 0x55e71d49e840 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f742259c640 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d498440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f741bfff640 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d4b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.647+0000 7f7422d9d640 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4c3690 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f741c004af0 tx=0x7f741c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47ebd9467a83ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.648+0000 7f740bfff640 1 -- 172.21.15.110:0/4023458832 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c0305c0 con 0x55e71d49e840 2024-04-25T00:06:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.648+0000 7f740bfff640 1 -- 172.21.15.110:0/4023458832 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741c054070 con 0x55e71d49e840 2024-04-25T00:06:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.648+0000 7f740bfff640 1 -- 172.21.15.110:0/4023458832 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c02f970 con 0x55e71d49e840 2024-04-25T00:06:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.649+0000 7f742b9d7740 1 -- 172.21.15.110:0/4023458832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 msgr2=0x55e71d4c3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.649+0000 7f742b9d7740 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4c3690 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f741c004af0 tx=0x7f741c043000 comp rx=0 tx=0).stop 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.649+0000 7f742b9d7740 1 -- 172.21.15.110:0/4023458832 shutdown_connections 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.649+0000 7f742b9d7740 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49e840 0x55e71d4c3690 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.649+0000 7f742b9d7740 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49de50 0x55e71d4b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.649+0000 7f742b9d7740 1 --2- 172.21.15.110:0/4023458832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d498440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.649+0000 7f742b9d7740 1 -- 172.21.15.110:0/4023458832 >> 172.21.15.110:0/4023458832 conn(0x55e71d49f620 msgr2=0x55e71d497d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.650+0000 7f742b9d7740 1 -- 172.21.15.110:0/4023458832 shutdown_connections 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.650+0000 7f742b9d7740 1 -- 172.21.15.110:0/4023458832 wait complete. 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.650+0000 7f742b9d7740 1 -- start start 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.650+0000 7f742b9d7740 1 Processor -- start 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d4b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d4b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742b9d7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742b9d7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f74241ad630 con 0x55e71d49de50 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d4b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39414/0 (socket says 172.21.15.110:39414) 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74241aeaa0 con 0x55e71d4994c0 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74241add00 con 0x55e71d49e840 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742259c640 1 -- 172.21.15.110:0/2226645087 learned_addr learned my addr 172.21.15.110:0/2226645087 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.651+0000 7f742b9d7740 10 client.0 _ll_register_callbacks cb 0x7f740c004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f7422d9d640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f741bfff640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f742259c640 1 -- 172.21.15.110:0/2226645087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 msgr2=0x55e71d4b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f742259c640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f742259c640 1 -- 172.21.15.110:0/2226645087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 msgr2=0x55e71d4b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f742259c640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f742259c640 1 -- 172.21.15.110:0/2226645087 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71d4b75b0 con 0x55e71d4994c0 2024-04-25T00:06:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f741bfff640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f7422d9d640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f742259c640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d4b5ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7414009610 tx=0x7f74140cc8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa3639aff33478a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f7409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f7409ffb640 1 -- 172.21.15.110:0/2226645087 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741409be30 con 0x55e71d4994c0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f7409ffb640 1 -- 172.21.15.110:0/2226645087 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7414010a30 con 0x55e71d4994c0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.652+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f741c054070 con 0x55e71d4994c0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f7409ffb640 1 -- 172.21.15.110:0/2226645087 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f741409b730 con 0x55e71d4994c0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f742b9d7740 10 client.25172 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e71d5233f0 con 0x55e71d4994c0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f7409ffb640 1 -- 172.21.15.110:0/2226645087 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741409fae0 con 0x55e71d4994c0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f7409ffb640 1 -- 172.21.15.110:0/2226645087 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f7414097510 con 0x55e71d4994c0 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f742b9d7740 10 client.25172 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f742b9d7740 10 client.25172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f742b9d7740 20 client.25172 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f742b9d7740 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e71d4a39b0 0x55e71d525d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f742b9d7740 4 client.25172 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:06:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f741c054070 con 0x55e71d4a39b0 2024-04-25T00:06:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.653+0000 7f741bfff640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e71d4a39b0 0x55e71d525d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.654+0000 7f7409ffb640 10 client.25172 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.654+0000 7f741bfff640 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e71d4a39b0 0x55e71d525d80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780fe61f575c6701 server_cookie=79cbd5300304f50d in_seq=0 out_seq=0 2024-04-25T00:06:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.672+0000 7f7409ffb640 1 -- 172.21.15.110:0/2226645087 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f741c054070 con 0x55e71d4a39b0 2024-04-25T00:06:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.672+0000 7f7409ffb640 10 client.25172 handle_command_reply: tid=0 2024-04-25T00:06:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.672+0000 7f742b9d7740 1 client.25172 shutdown 2024-04-25T00:06:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.672+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 msgr2=0x55e71d4b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.672+0000 7f742b9d7740 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d4b5ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7414009610 tx=0x7f74140cc8a0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 shutdown_connections 2024-04-25T00:06:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e71d4a39b0 0x55e71d525d80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e71d49e840 0x55e71d4b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e71d49de50 0x55e71d4b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 1 --2- 172.21.15.110:0/2226645087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e71d4994c0 0x55e71d4b5ac0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 >> 172.21.15.110:0/2226645087 conn(0x55e71d49f620 msgr2=0x55e71d4985c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f7409ffb640 0 client.25172 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 shutdown_connections 2024-04-25T00:06:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 1 -- 172.21.15.110:0/2226645087 wait complete. 2024-04-25T00:06:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.673+0000 7f742b9d7740 20 client.25172 trim_cache size 0 max 16384 2024-04-25T00:06:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:06:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:06:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:06:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:06:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 -- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f74100a2e20 msgr2=0x7f74100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f74100a2e20 0x7f74100a52e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55e71d4aee40 tx=0x7f741c0308a0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 -- 172.21.15.110:0/1787243305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410c190 msgr2=0x7f74241a22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410c190 0x7f74241a22d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f741400d950 tx=0x7f741400de10 comp rx=0 tx=0).stop 2024-04-25T00:06:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 -- 172.21.15.110:0/1787243305 shutdown_connections 2024-04-25T00:06:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f74100a2e20 0x7f74100a52e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742410dd40 0x7f74241a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f742410d380 0x7f74241a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 --2- 172.21.15.110:0/1787243305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f742410c190 0x7f74241a22d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 -- 172.21.15.110:0/1787243305 >> 172.21.15.110:0/1787243305 conn(0x7f742406c360 msgr2=0x7f7424110680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 -- 172.21.15.110:0/1787243305 shutdown_connections 2024-04-25T00:06:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:04.674+0000 7f7428cca640 1 -- 172.21.15.110:0/1787243305 wait complete. 2024-04-25T00:06:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:05 smithi188 ceph-mon[33285]: pgmap v472: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 1.10k op/s 2024-04-25T00:06:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:05 smithi110 ceph-mon[29187]: pgmap v472: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 1.10k op/s 2024-04-25T00:06:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:05 smithi173 ceph-mon[32402]: pgmap v472: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 1.10k op/s 2024-04-25T00:06:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 180.0 s, 824.8 tps, lat 121.346 ms stddev 14.411 2024-04-25T00:06:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:07 smithi188 ceph-mon[33285]: pgmap v473: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 1.01k op/s 2024-04-25T00:06:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:07 smithi110 ceph-mon[29187]: pgmap v473: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 1.01k op/s 2024-04-25T00:06:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:07 smithi173 ceph-mon[32402]: pgmap v473: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 1.01k op/s 2024-04-25T00:06:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:09 smithi188 ceph-mon[33285]: pgmap v474: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 1.19k op/s 2024-04-25T00:06:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:09 smithi110 ceph-mon[29187]: pgmap v474: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 1.19k op/s 2024-04-25T00:06:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:09 smithi173 ceph-mon[32402]: pgmap v474: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 1.19k op/s 2024-04-25T00:06:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 185.0 s, 861.0 tps, lat 116.199 ms stddev 10.430 2024-04-25T00:06:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:11 smithi188 ceph-mon[33285]: pgmap v475: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:11 smithi110 ceph-mon[29187]: pgmap v475: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:11 smithi173 ceph-mon[32402]: pgmap v475: 97 pgs: 97 active+clean; 1022 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:13 smithi188 ceph-mon[33285]: pgmap v476: 97 pgs: 97 active+clean; 1022 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 1.11k op/s 2024-04-25T00:06:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:13 smithi110 ceph-mon[29187]: pgmap v476: 97 pgs: 97 active+clean; 1022 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 1.11k op/s 2024-04-25T00:06:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:13 smithi173 ceph-mon[32402]: pgmap v476: 97 pgs: 97 active+clean; 1022 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 1.11k op/s 2024-04-25T00:06:16.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:15 smithi188 ceph-mon[33285]: pgmap v477: 97 pgs: 97 active+clean; 1023 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 1.11k op/s 2024-04-25T00:06:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:15 smithi110 ceph-mon[29187]: pgmap v477: 97 pgs: 97 active+clean; 1023 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 1.11k op/s 2024-04-25T00:06:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:15 smithi173 ceph-mon[32402]: pgmap v477: 97 pgs: 97 active+clean; 1023 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 1.11k op/s 2024-04-25T00:06:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 190.0 s, 871.8 tps, lat 114.855 ms stddev 23.030 2024-04-25T00:06:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.791+0000 7efe141dc640 1 -- 172.21.15.110:0/3457326039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0c1029c0 msgr2=0x7efe0c102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 --2- 172.21.15.110:0/3457326039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0c1029c0 0x7efe0c102e40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009900 tx=0x7efdfc02a410 comp rx=0 tx=0).stop 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 -- 172.21.15.110:0/3457326039 shutdown_connections 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 --2- 172.21.15.110:0/3457326039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0c103380 0x7efe0c109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 --2- 172.21.15.110:0/3457326039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0c1029c0 0x7efe0c102e40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 --2- 172.21.15.110:0/3457326039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0c1017d0 0x7efe0c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 -- 172.21.15.110:0/3457326039 >> 172.21.15.110:0/3457326039 conn(0x7efe0c0fcd60 msgr2=0x7efe0c0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 -- 172.21.15.110:0/3457326039 shutdown_connections 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 -- 172.21.15.110:0/3457326039 wait complete. 2024-04-25T00:06:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.792+0000 7efe141dc640 1 Processor -- start 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe141dc640 1 -- start start 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe141dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0c1017d0 0x7efe0c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe141dc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0c1029c0 0x7efe0c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe141dc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0c103380 0x7efe0c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe141dc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c10d300 con 0x7efe0c1017d0 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe141dc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c10d180 con 0x7efe0c103380 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe141dc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c10d480 con 0x7efe0c1029c0 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11f78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0c1017d0 0x7efe0c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11777640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0c1029c0 0x7efe0c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11777640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0c1029c0 0x7efe0c073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37714/0 (socket says 172.21.15.110:37714) 2024-04-25T00:06:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11777640 1 -- 172.21.15.110:0/1373700329 learned_addr learned my addr 172.21.15.110:0/1373700329 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe12779640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0c103380 0x7efe0c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11777640 1 -- 172.21.15.110:0/1373700329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0c103380 msgr2=0x7efe0c074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11777640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0c103380 0x7efe0c074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11777640 1 -- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0c1017d0 msgr2=0x7efe0c0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11777640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0c1017d0 0x7efe0c0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.793+0000 7efe11777640 1 -- 172.21.15.110:0/1373700329 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c074900 con 0x7efe0c1029c0 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.794+0000 7efe11777640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0c1029c0 0x7efe0c073b30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005940 tx=0x7efdfc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.794+0000 7efe02ffd640 1 -- 172.21.15.110:0/1373700329 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc03f030 con 0x7efe0c1029c0 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.794+0000 7efe02ffd640 1 -- 172.21.15.110:0/1373700329 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdfc0318f0 con 0x7efe0c1029c0 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.794+0000 7efe02ffd640 1 -- 172.21.15.110:0/1373700329 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc035070 con 0x7efe0c1029c0 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.794+0000 7efe141dc640 1 -- 172.21.15.110:0/1373700329 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c1a19b0 con 0x7efe0c1029c0 2024-04-25T00:06:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.794+0000 7efe141dc640 1 -- 172.21.15.110:0/1373700329 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c1a1eb0 con 0x7efe0c1029c0 2024-04-25T00:06:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.797+0000 7efe02ffd640 1 -- 172.21.15.110:0/1373700329 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efdfc042030 con 0x7efe0c1029c0 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.797+0000 7efe16fc9740 1 Processor -- start 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe16fc9740 1 -- start start 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c688f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6894340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c6896e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe16fc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e0c67f2cb0 con 0x55e0c688ee50 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe16fc9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e0c679b5b0 con 0x55e0c68949b0 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe16fc9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e0c67eec70 con 0x55e0c688f840 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe12779640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c688f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe11f78640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6894340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe11f78640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6894340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37722/0 (socket says 172.21.15.110:37722) 2024-04-25T00:06:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe11f78640 1 -- 172.21.15.110:0/3750935147 learned_addr learned my addr 172.21.15.110:0/3750935147 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.798+0000 7efe11777640 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c6896e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe02ffd640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efdf80a2c30 0x7efdf80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe11f78640 1 -- 172.21.15.110:0/3750935147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 msgr2=0x55e0c6896e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe11f78640 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c6896e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe11f78640 1 -- 172.21.15.110:0/3750935147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 msgr2=0x55e0c688f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe11f78640 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c688f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe11f78640 1 -- 172.21.15.110:0/3750935147 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c6507210 con 0x55e0c688f840 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe12779640 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c688f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe12779640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efdf80a2c30 0x7efdf80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe02ffd640 1 -- 172.21.15.110:0/1373700329 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efdfc0ef4a0 con 0x7efe0c1029c0 2024-04-25T00:06:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efe11f78640 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6894340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe08009ca0 tx=0x7efe08011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=261956cc7abe66fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.799+0000 7efdf7fff640 1 -- 172.21.15.110:0/3750935147 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08004220 con 0x55e0c688f840 2024-04-25T00:06:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.800+0000 7efdf7fff640 1 -- 172.21.15.110:0/3750935147 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe080043c0 con 0x55e0c688f840 2024-04-25T00:06:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.800+0000 7efdf7fff640 1 -- 172.21.15.110:0/3750935147 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe080198b0 con 0x55e0c688f840 2024-04-25T00:06:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.800+0000 7efe12779640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efdf80a2c30 0x7efdf80a50f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55e0c68900a0 tx=0x7efe04014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 -- 172.21.15.110:0/3750935147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 msgr2=0x55e0c6894340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6894340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe08009ca0 tx=0x7efe08011810 comp rx=0 tx=0).stop 2024-04-25T00:06:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 -- 172.21.15.110:0/3750935147 shutdown_connections 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c6896e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6894340 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 --2- 172.21.15.110:0/3750935147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c688f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 -- 172.21.15.110:0/3750935147 >> 172.21.15.110:0/3750935147 conn(0x55e0c688a4c0 msgr2=0x55e0c688a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 -- 172.21.15.110:0/3750935147 shutdown_connections 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.801+0000 7efe16fc9740 1 -- 172.21.15.110:0/3750935147 wait complete. 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 1 -- start start 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 1 Processor -- start 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6899300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6899840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c68a0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e0c6897960 con 0x55e0c688ee50 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e0c67aeb50 con 0x55e0c68949b0 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e0c6897ae0 con 0x55e0c688f840 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.802+0000 7efe16fc9740 10 client.0 _ll_register_callbacks cb 0x55e0c67e9ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c68a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe11777640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6899840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c68a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:60472/0 (socket says 172.21.15.110:60472) 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe11f78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6899300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 -- 172.21.15.110:0/2131102788 learned_addr learned my addr 172.21.15.110:0/2131102788 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 -- 172.21.15.110:0/2131102788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 msgr2=0x55e0c6899840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6899840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 -- 172.21.15.110:0/2131102788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 msgr2=0x55e0c6899300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6899300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 -- 172.21.15.110:0/2131102788 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c65d3620 con 0x55e0c68949b0 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe11777640 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6899840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efe12779640 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c68a0880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe04009ce0 tx=0x7efe04009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f28fa2d91ddde8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.803+0000 7efdf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efdf57fa640 1 -- 172.21.15.110:0/2131102788 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0401c950 con 0x55e0c68949b0 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efdf57fa640 1 -- 172.21.15.110:0/2131102788 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efe0401caf0 con 0x55e0c68949b0 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 1 -- 172.21.15.110:0/2131102788 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe080043c0 con 0x55e0c68949b0 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efdf57fa640 1 -- 172.21.15.110:0/2131102788 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efe0401ccf0 con 0x55e0c68949b0 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 10 client.25184 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 1 -- 172.21.15.110:0/2131102788 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e0c68a3c60 con 0x55e0c68949b0 2024-04-25T00:06:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efdf57fa640 1 -- 172.21.15.110:0/2131102788 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe080043c0 con 0x55e0c68949b0 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efdf57fa640 1 -- 172.21.15.110:0/2131102788 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7efe04025020 con 0x55e0c68949b0 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 10 client.25184 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 20 client.25184 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e0c68a53a0 0x55e0c68a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 4 client.25184 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.804+0000 7efe16fc9740 1 -- 172.21.15.110:0/2131102788 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0c68001f0 con 0x55e0c68a53a0 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.805+0000 7efe11f78640 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e0c68a53a0 0x55e0c68a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.805+0000 7efe11f78640 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e0c68a53a0 0x55e0c68a7810 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bcb23a85c32d12 server_cookie=2426cb974d65c707 in_seq=0 out_seq=0 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.805+0000 7efdf57fa640 10 client.25184 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efdf57fa640 1 -- 172.21.15.110:0/2131102788 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0c68001f0 con 0x55e0c68a53a0 2024-04-25T00:06:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efdf57fa640 10 client.25184 handle_command_reply: tid=0 2024-04-25T00:06:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 client.25184 shutdown 2024-04-25T00:06:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 -- 172.21.15.110:0/2131102788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 msgr2=0x55e0c68a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c68a0880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe04009ce0 tx=0x7efe04009500 comp rx=0 tx=0).stop 2024-04-25T00:06:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 -- 172.21.15.110:0/2131102788 shutdown_connections 2024-04-25T00:06:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e0c68a53a0 0x55e0c68a7810 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c68949b0 0x55e0c68a0880 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efdf57fa640 0 client.25184 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c688f840 0x55e0c6899840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2131102788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6899300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.806+0000 7efe16fc9740 1 -- 172.21.15.110:0/2131102788 >> 172.21.15.110:0/2131102788 conn(0x55e0c688a4c0 msgr2=0x55e0c68895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.807+0000 7efe16fc9740 1 -- 172.21.15.110:0/2131102788 shutdown_connections 2024-04-25T00:06:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.807+0000 7efe16fc9740 1 -- 172.21.15.110:0/2131102788 wait complete. 2024-04-25T00:06:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.807+0000 7efe16fc9740 20 client.25184 trim_cache size 0 max 16384 2024-04-25T00:06:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe16fc9740 1 Processor -- start 2024-04-25T00:06:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe16fc9740 1 -- start start 2024-04-25T00:06:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6890ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c68a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 0x55e0c68911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe16fc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1ad520 con 0x55e0c688ee50 2024-04-25T00:06:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe16fc9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1aadc0 con 0x55e0c688f840 2024-04-25T00:06:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe16fc9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1a8c10 con 0x55e0c68949b0 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11f78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6890ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11777640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c68a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11f78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6890ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44780/0 (socket says 172.21.15.110:44780) 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11f78640 1 -- 172.21.15.110:0/2628179960 learned_addr learned my addr 172.21.15.110:0/2628179960 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe12779640 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 0x55e0c68911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11f78640 1 -- 172.21.15.110:0/2628179960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 msgr2=0x55e0c68911e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11f78640 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 0x55e0c68911e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11f78640 1 -- 172.21.15.110:0/2628179960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 msgr2=0x55e0c68a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11f78640 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c68a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11f78640 1 -- 172.21.15.110:0/2628179960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c68900d0 con 0x55e0c688ee50 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.830+0000 7efe11777640 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c68a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.831+0000 7efe11f78640 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6890ca0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efe08009bc0 tx=0x7efe08011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61eb11f41824b937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.831+0000 7efdf77fe640 1 -- 172.21.15.110:0/2628179960 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0802f070 con 0x55e0c688ee50 2024-04-25T00:06:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.831+0000 7efdf77fe640 1 -- 172.21.15.110:0/2628179960 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe080052d0 con 0x55e0c688ee50 2024-04-25T00:06:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.831+0000 7efdf77fe640 1 -- 172.21.15.110:0/2628179960 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0802b7f0 con 0x55e0c688ee50 2024-04-25T00:06:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 -- 172.21.15.110:0/2628179960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 msgr2=0x55e0c6890ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6890ca0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efe08009bc0 tx=0x7efe08011810 comp rx=0 tx=0).stop 2024-04-25T00:06:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 -- 172.21.15.110:0/2628179960 shutdown_connections 2024-04-25T00:06:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 0x55e0c68911e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c68a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 --2- 172.21.15.110:0/2628179960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c6890ca0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 -- 172.21.15.110:0/2628179960 >> 172.21.15.110:0/2628179960 conn(0x55e0c688a4c0 msgr2=0x55e0c688a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 -- 172.21.15.110:0/2628179960 shutdown_connections 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 -- 172.21.15.110:0/2628179960 wait complete. 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 -- start start 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.832+0000 7efe16fc9740 1 Processor -- start 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c689d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c689de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe16fc9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 0x55e0c689ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe16fc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1a9280 con 0x55e0c688ee50 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe16fc9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1aa6f0 con 0x55e0c688f840 2024-04-25T00:06:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe16fc9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c1a9950 con 0x55e0c68949b0 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c689d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe16fc9740 10 client.0 _ll_register_callbacks cb 0x7efe04009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c689d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44782/0 (socket says 172.21.15.110:44782) 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 -- 172.21.15.110:0/275414160 learned_addr learned my addr 172.21.15.110:0/275414160 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11777640 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c689de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe12779640 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 0x55e0c689ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 -- 172.21.15.110:0/275414160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 msgr2=0x55e0c689ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 0x55e0c689ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 -- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 msgr2=0x55e0c689de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c689de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 -- 172.21.15.110:0/275414160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c689f430 con 0x55e0c688ee50 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11777640 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c689de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efe11f78640 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c689d8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efe08009250 tx=0x7efe08005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4299bdf3ecce10a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efdf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.833+0000 7efdf57fa640 1 -- 172.21.15.110:0/275414160 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08038030 con 0x55e0c688ee50 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.834+0000 7efe16fc9740 1 -- 172.21.15.110:0/275414160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe080052d0 con 0x55e0c688ee50 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.834+0000 7efdf57fa640 1 -- 172.21.15.110:0/275414160 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efe080035f0 con 0x55e0c688ee50 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.834+0000 7efdf57fa640 1 -- 172.21.15.110:0/275414160 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efe0802f040 con 0x55e0c688ee50 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.834+0000 7efe16fc9740 10 client.15609 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.834+0000 7efe16fc9740 1 -- 172.21.15.110:0/275414160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e0c68a0740 con 0x55e0c688ee50 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.834+0000 7efdf57fa640 1 -- 172.21.15.110:0/275414160 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe080052d0 con 0x55e0c688ee50 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.834+0000 7efdf57fa640 1 -- 172.21.15.110:0/275414160 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7efe0802b930 con 0x55e0c688ee50 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.834+0000 7efe16fc9740 10 client.15609 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.835+0000 7efe16fc9740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.835+0000 7efe16fc9740 20 client.15609 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.835+0000 7efe16fc9740 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e0c690bba0 0x55e0c690df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.835+0000 7efe16fc9740 4 client.15609 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:06:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.835+0000 7efe16fc9740 1 -- 172.21.15.110:0/275414160 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0c67fa540 con 0x55e0c690bba0 2024-04-25T00:06:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.835+0000 7efe11777640 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e0c690bba0 0x55e0c690df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.836+0000 7efdf57fa640 10 client.15609 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:17.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.836+0000 7efe11777640 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e0c690bba0 0x55e0c690df90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f2b89fa0a39878 server_cookie=1aa5acd1fb53c7b0 in_seq=0 out_seq=0 2024-04-25T00:06:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.851+0000 7efdf57fa640 1 -- 172.21.15.110:0/275414160 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0c67fa540 con 0x55e0c690bba0 2024-04-25T00:06:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.851+0000 7efdf57fa640 10 client.15609 handle_command_reply: tid=0 2024-04-25T00:06:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.851+0000 7efe16fc9740 1 client.15609 shutdown 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.851+0000 7efe16fc9740 1 -- 172.21.15.110:0/275414160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 msgr2=0x55e0c689d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.851+0000 7efe16fc9740 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c689d8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efe08009250 tx=0x7efe08005dc0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 1 -- 172.21.15.110:0/275414160 shutdown_connections 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e0c690bba0 0x55e0c690df90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0c68949b0 0x55e0c689ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efdf57fa640 0 client.15609 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e0c688f840 0x55e0c689de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 1 --2- 172.21.15.110:0/275414160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0c688ee50 0x55e0c689d8f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 1 -- 172.21.15.110:0/275414160 >> 172.21.15.110:0/275414160 conn(0x55e0c688a4c0 msgr2=0x55e0c6899250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 1 -- 172.21.15.110:0/275414160 shutdown_connections 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 1 -- 172.21.15.110:0/275414160 wait complete. 2024-04-25T00:06:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.852+0000 7efe16fc9740 20 client.15609 trim_cache size 0 max 16384 2024-04-25T00:06:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:06:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:06:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:06:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 -- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efdf80a2c30 msgr2=0x7efdf80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efdf80a2c30 0x7efdf80a50f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55e0c68900a0 tx=0x7efe04014040 comp rx=0 tx=0).stop 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 -- 172.21.15.110:0/1373700329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0c1029c0 msgr2=0x7efe0c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0c1029c0 0x7efe0c073b30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005940 tx=0x7efdfc0042f0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 -- 172.21.15.110:0/1373700329 shutdown_connections 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efdf80a2c30 0x7efdf80a50f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0c103380 0x7efe0c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe0c1029c0 0x7efe0c073b30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 --2- 172.21.15.110:0/1373700329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe0c1017d0 0x7efe0c0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 -- 172.21.15.110:0/1373700329 >> 172.21.15.110:0/1373700329 conn(0x7efe0c0fcd60 msgr2=0x7efe0c105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 -- 172.21.15.110:0/1373700329 shutdown_connections 2024-04-25T00:06:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:17.853+0000 7efe141dc640 1 -- 172.21.15.110:0/1373700329 wait complete. 2024-04-25T00:06:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:17 smithi188 ceph-mon[33285]: pgmap v478: 97 pgs: 97 active+clean; 1023 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:17 smithi110 ceph-mon[29187]: pgmap v478: 97 pgs: 97 active+clean; 1023 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:17 smithi173 ceph-mon[32402]: pgmap v478: 97 pgs: 97 active+clean; 1023 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:19 smithi188 ceph-mon[33285]: pgmap v479: 97 pgs: 97 active+clean; 1023 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-04-25T00:06:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:19 smithi110 ceph-mon[29187]: pgmap v479: 97 pgs: 97 active+clean; 1023 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-04-25T00:06:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:19 smithi173 ceph-mon[32402]: pgmap v479: 97 pgs: 97 active+clean; 1023 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-04-25T00:06:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 195.0 s, 800.2 tps, lat 124.804 ms stddev 83.801 2024-04-25T00:06:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:21 smithi188 ceph-mon[33285]: pgmap v480: 97 pgs: 97 active+clean; 1023 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.9 MiB/s wr, 1.08k op/s 2024-04-25T00:06:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:21 smithi110 ceph-mon[29187]: pgmap v480: 97 pgs: 97 active+clean; 1023 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.9 MiB/s wr, 1.08k op/s 2024-04-25T00:06:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:21 smithi173 ceph-mon[32402]: pgmap v480: 97 pgs: 97 active+clean; 1023 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.9 MiB/s wr, 1.08k op/s 2024-04-25T00:06:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:23 smithi188 ceph-mon[33285]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 1.20k op/s 2024-04-25T00:06:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:23 smithi110 ceph-mon[29187]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 1.20k op/s 2024-04-25T00:06:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:23 smithi173 ceph-mon[32402]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 1.20k op/s 2024-04-25T00:06:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:25 smithi188 ceph-mon[33285]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:25 smithi110 ceph-mon[29187]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:25 smithi173 ceph-mon[32402]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 200.0 s, 783.4 tps, lat 127.787 ms stddev 18.631 2024-04-25T00:06:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:27 smithi188 ceph-mon[33285]: pgmap v483: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 939 op/s 2024-04-25T00:06:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:27 smithi110 ceph-mon[29187]: pgmap v483: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 939 op/s 2024-04-25T00:06:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:27 smithi173 ceph-mon[32402]: pgmap v483: 97 pgs: 97 active+clean; 1.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 939 op/s 2024-04-25T00:06:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:29 smithi188 ceph-mon[33285]: pgmap v484: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-04-25T00:06:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:29 smithi110 ceph-mon[29187]: pgmap v484: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-04-25T00:06:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:29 smithi173 ceph-mon[32402]: pgmap v484: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-04-25T00:06:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.983+0000 7f0c34a12640 1 -- 172.21.15.110:0/1964479663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30076100 msgr2=0x7f0c30076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.983+0000 7f0c34a12640 1 --2- 172.21.15.110:0/1964479663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30076100 0x7f0c30076520 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009860 tx=0x7f0c2002a340 comp rx=0 tx=0).stop 2024-04-25T00:06:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.983+0000 7f0c34a12640 1 -- 172.21.15.110:0/1964479663 shutdown_connections 2024-04-25T00:06:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.984+0000 7f0c34a12640 1 --2- 172.21.15.110:0/1964479663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c30077ce0 0x7f0c30078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.984+0000 7f0c34a12640 1 --2- 172.21.15.110:0/1964479663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c30077340 0x7f0c300777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.984+0000 7f0c34a12640 1 --2- 172.21.15.110:0/1964479663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30076100 0x7f0c30076520 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.984+0000 7f0c34a12640 1 -- 172.21.15.110:0/1964479663 >> 172.21.15.110:0/1964479663 conn(0x7f0c300fef60 msgr2=0x7f0c30101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.984+0000 7f0c34a12640 1 -- 172.21.15.110:0/1964479663 shutdown_connections 2024-04-25T00:06:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.984+0000 7f0c34a12640 1 -- 172.21.15.110:0/1964479663 wait complete. 2024-04-25T00:06:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.985+0000 7f0c34a12640 1 Processor -- start 2024-04-25T00:06:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.985+0000 7f0c34a12640 1 -- start start 2024-04-25T00:06:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.985+0000 7f0c34a12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c30077340 0x7f0c3019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c34a12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30077ce0 0x7f0c3019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c34a12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c301a36e0 0x7f0c301a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c30077340 0x7f0c3019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c30077340 0x7f0c3019e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:36188/0 (socket says 172.21.15.110:36188) 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2e59c640 1 -- 172.21.15.110:0/419821573 learned_addr learned my addr 172.21.15.110:0/419821573 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c27fff640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30077ce0 0x7f0c3019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c301a36e0 0x7f0c301a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30113910 con 0x7f0c30077ce0 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30113790 con 0x7f0c301a36e0 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30113a90 con 0x7f0c30077340 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2ed9d640 1 -- 172.21.15.110:0/419821573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c30077340 msgr2=0x7f0c3019e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c30077340 0x7f0c3019e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2ed9d640 1 -- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30077ce0 msgr2=0x7f0c3019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30077ce0 0x7f0c3019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c2ed9d640 1 -- 172.21.15.110:0/419821573 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c301a6140 con 0x7f0c301a36e0 2024-04-25T00:06:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.986+0000 7f0c27fff640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30077ce0 0x7f0c3019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.987+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c301a36e0 0x7f0c301a5ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800efc0 tx=0x7f0c2800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.987+0000 7f0c277fe640 1 -- 172.21.15.110:0/419821573 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28003970 con 0x7f0c301a36e0 2024-04-25T00:06:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.987+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c301a6430 con 0x7f0c301a36e0 2024-04-25T00:06:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.987+0000 7f0c277fe640 1 -- 172.21.15.110:0/419821573 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c28004020 con 0x7f0c301a36e0 2024-04-25T00:06:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.987+0000 7f0c277fe640 1 -- 172.21.15.110:0/419821573 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2800b940 con 0x7f0c301a36e0 2024-04-25T00:06:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.987+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c301a6980 con 0x7f0c301a36e0 2024-04-25T00:06:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.990+0000 7f0c277fe640 1 -- 172.21.15.110:0/419821573 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0c2801d020 con 0x7f0c301a36e0 2024-04-25T00:06:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c377f9740 1 Processor -- start 2024-04-25T00:06:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c277fe640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c1c0a2cb0 0x7f0c1c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c377f9740 1 -- start start 2024-04-25T00:06:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c277fe640 1 -- 172.21.15.110:0/419821573 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0c280cee00 con 0x7f0c301a36e0 2024-04-25T00:06:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002431e50 0x55d002432270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002432840 0x55d002437340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 0x55d002439e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c377f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d002395cb0 con 0x55d002432840 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c377f9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d00233e5b0 con 0x55d002431e50 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c377f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d002391c70 con 0x55d0024379b0 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c2e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002431e50 0x55d002432270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c2e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002431e50 0x55d002432270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:59376/0 (socket says 172.21.15.110:59376) 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c2e59c640 1 -- 172.21.15.110:0/1958104402 learned_addr learned my addr 172.21.15.110:0/1958104402 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.991+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 0x55d002439e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c27fff640 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002432840 0x55d002437340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c27fff640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c1c0a2cb0 0x7f0c1c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c2e59c640 1 -- 172.21.15.110:0/1958104402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 msgr2=0x55d002439e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c2e59c640 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 0x55d002439e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c2e59c640 1 -- 172.21.15.110:0/1958104402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002432840 msgr2=0x55d002437340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c2e59c640 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002432840 0x55d002437340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c2e59c640 1 -- 172.21.15.110:0/1958104402 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0020aa210 con 0x55d002431e50 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c2e59c640 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002431e50 0x55d002432270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000c00 tx=0x7f0c20005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48a1946b2da8f907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c24ff9640 1 -- 172.21.15.110:0/1958104402 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20037040 con 0x55d002431e50 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c24ff9640 1 -- 172.21.15.110:0/1958104402 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c20038040 con 0x55d002431e50 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.992+0000 7f0c24ff9640 1 -- 172.21.15.110:0/1958104402 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20003830 con 0x55d002431e50 2024-04-25T00:06:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.993+0000 7f0c27fff640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c1c0a2cb0 0x7f0c1c0a5170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55d002432c40 tx=0x7f0c18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.993+0000 7f0c377f9740 1 -- 172.21.15.110:0/1958104402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002431e50 msgr2=0x55d002432270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.993+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002431e50 0x55d002432270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000c00 tx=0x7f0c20005b20 comp rx=0 tx=0).stop 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 -- 172.21.15.110:0/1958104402 shutdown_connections 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 0x55d002439e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002432840 0x55d002437340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1958104402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002431e50 0x55d002432270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 -- 172.21.15.110:0/1958104402 >> 172.21.15.110:0/1958104402 conn(0x55d00242d4c0 msgr2=0x55d00242d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 -- 172.21.15.110:0/1958104402 shutdown_connections 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 -- 172.21.15.110:0/1958104402 wait complete. 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 -- start start 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.994+0000 7f0c377f9740 1 Processor -- start 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002431e50 0x55d00243c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d00243c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d0024379b0 0x55d0024435b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c377f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d00243a900 con 0x55d0024379b0 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c377f9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d002351b50 con 0x55d002432840 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c377f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d00243aa80 con 0x55d002431e50 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c377f9740 10 client.0 _ll_register_callbacks cb 0x55d00238cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d00243c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c2ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d0024379b0 0x55d0024435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c2e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002431e50 0x55d00243c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d00243c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:59378/0 (socket says 172.21.15.110:59378) 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 -- 172.21.15.110:0/1616224466 learned_addr learned my addr 172.21.15.110:0/1616224466 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 -- 172.21.15.110:0/1616224466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002431e50 msgr2=0x55d00243c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002431e50 0x55d00243c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 -- 172.21.15.110:0/1616224466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d0024379b0 msgr2=0x55d0024435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d0024379b0 0x55d0024435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 -- 172.21.15.110:0/1616224466 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d002176620 con 0x55d002432840 2024-04-25T00:06:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d0024379b0 0x55d0024435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c27fff640 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d00243c570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800fb60 tx=0x7f0c18004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c643b1aff394c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.995+0000 7f0c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:06:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.996+0000 7f0c15ffb640 1 -- 172.21.15.110:0/1616224466 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c180044c0 con 0x55d002432840 2024-04-25T00:06:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.996+0000 7f0c15ffb640 1 -- 172.21.15.110:0/1616224466 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0c18003b70 con 0x55d002432840 2024-04-25T00:06:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.996+0000 7f0c377f9740 1 -- 172.21.15.110:0/1616224466 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20038040 con 0x55d002432840 2024-04-25T00:06:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.996+0000 7f0c15ffb640 1 -- 172.21.15.110:0/1616224466 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0c1800fe60 con 0x55d002432840 2024-04-25T00:06:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.996+0000 7f0c377f9740 10 client.25202 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.996+0000 7f0c377f9740 1 -- 172.21.15.110:0/1616224466 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d0024469e0 con 0x55d002432840 2024-04-25T00:06:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.996+0000 7f0c15ffb640 1 -- 172.21.15.110:0/1616224466 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c20038040 con 0x55d002432840 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.996+0000 7f0c15ffb640 1 -- 172.21.15.110:0/1616224466 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f0c18027070 con 0x55d002432840 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c377f9740 10 client.25202 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c377f9740 10 client.25202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c377f9740 20 client.25202 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d0024480c0 0x55d00244a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c377f9740 4 client.25202 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c377f9740 1 -- 172.21.15.110:0/1616224466 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0023a31f0 con 0x55d0024480c0 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c2e59c640 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d0024480c0 0x55d00244a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c15ffb640 10 client.25202 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.997+0000 7f0c2e59c640 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d0024480c0 0x55d00244a550 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d17c74c6bdbefde5 server_cookie=3528eae3ce074bbc in_seq=0 out_seq=0 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.998+0000 7f0c15ffb640 1 -- 172.21.15.110:0/1616224466 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0023a31f0 con 0x55d0024480c0 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.998+0000 7f0c15ffb640 10 client.25202 handle_command_reply: tid=0 2024-04-25T00:06:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.998+0000 7f0c377f9740 1 client.25202 shutdown 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.998+0000 7f0c377f9740 1 -- 172.21.15.110:0/1616224466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 msgr2=0x55d00243c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d00243c570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800fb60 tx=0x7f0c18004730 comp rx=0 tx=0).stop 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 -- 172.21.15.110:0/1616224466 shutdown_connections 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d0024480c0 0x55d00244a550 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d0024379b0 0x55d0024435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d00243c570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 --2- 172.21.15.110:0/1616224466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002431e50 0x55d00243c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 -- 172.21.15.110:0/1616224466 >> 172.21.15.110:0/1616224466 conn(0x55d00242d4c0 msgr2=0x55d00242c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c15ffb640 0 client.25202 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 -- 172.21.15.110:0/1616224466 shutdown_connections 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 1 -- 172.21.15.110:0/1616224466 wait complete. 2024-04-25T00:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:30.999+0000 7f0c377f9740 20 client.25202 trim_cache size 0 max 16384 2024-04-25T00:06:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c377f9740 1 Processor -- start 2024-04-25T00:06:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c377f9740 1 -- start start 2024-04-25T00:06:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 0x55d00243c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002432840 0x55d00243c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0024379b0 0x55d002443570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c377f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301b1e90 con 0x55d002431e50 2024-04-25T00:06:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c377f9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301af730 con 0x55d0024379b0 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c377f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301ad580 con 0x55d002432840 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2ed9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0024379b0 0x55d002443570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2ed9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0024379b0 0x55d002443570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:59386/0 (socket says 172.21.15.110:59386) 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2ed9d640 1 -- 172.21.15.110:0/224647330 learned_addr learned my addr 172.21.15.110:0/224647330 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c27fff640 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002432840 0x55d00243c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2e59c640 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 0x55d00243c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2ed9d640 1 -- 172.21.15.110:0/224647330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002432840 msgr2=0x55d00243c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002432840 0x55d00243c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2ed9d640 1 -- 172.21.15.110:0/224647330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 msgr2=0x55d00243c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 0x55d00243c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.022+0000 7f0c2ed9d640 1 -- 172.21.15.110:0/224647330 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d002436d40 con 0x55d0024379b0 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.023+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0024379b0 0x55d002443570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009830 tx=0x7f0c280ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a3ea9d32706bb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.023+0000 7f0c17fff640 1 -- 172.21.15.110:0/224647330 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2809b690 con 0x55d0024379b0 2024-04-25T00:06:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.023+0000 7f0c17fff640 1 -- 172.21.15.110:0/224647330 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c2809fa40 con 0x55d0024379b0 2024-04-25T00:06:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.023+0000 7f0c17fff640 1 -- 172.21.15.110:0/224647330 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c280a3e70 con 0x55d0024379b0 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 -- 172.21.15.110:0/224647330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0024379b0 msgr2=0x55d002443570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0024379b0 0x55d002443570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009830 tx=0x7f0c280ce040 comp rx=0 tx=0).stop 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 -- 172.21.15.110:0/224647330 shutdown_connections 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0024379b0 0x55d002443570 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d002432840 0x55d00243c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 --2- 172.21.15.110:0/224647330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 0x55d00243c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 -- 172.21.15.110:0/224647330 >> 172.21.15.110:0/224647330 conn(0x55d00242d4c0 msgr2=0x55d0024370c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 -- 172.21.15.110:0/224647330 shutdown_connections 2024-04-25T00:06:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 -- 172.21.15.110:0/224647330 wait complete. 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.024+0000 7f0c377f9740 1 -- start start 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c377f9740 1 Processor -- start 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 0x55d0024404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d002440a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c377f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 0x55d0024ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c377f9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301adbf0 con 0x55d002431e50 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c377f9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301af060 con 0x55d002432840 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c377f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301ae2c0 con 0x55d0024379b0 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c377f9740 10 client.0 _ll_register_callbacks cb 0x7f0c180115e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d002440a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d002440a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:59400/0 (socket says 172.21.15.110:59400) 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c2e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 0x55d0024404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c27fff640 1 -- 172.21.15.110:0/734514074 learned_addr learned my addr 172.21.15.110:0/734514074 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c2ed9d640 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 0x55d0024ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c27fff640 1 -- 172.21.15.110:0/734514074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 msgr2=0x55d0024ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c27fff640 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 0x55d0024ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c27fff640 1 -- 172.21.15.110:0/734514074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 msgr2=0x55d0024404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.025+0000 7f0c27fff640 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 0x55d0024404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c27fff640 1 -- 172.21.15.110:0/734514074 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0024acc00 con 0x55d002432840 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c27fff640 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d002440a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c18029700 tx=0x7f0c18003d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2388ff37603392a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c15ffb640 1 -- 172.21.15.110:0/734514074 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18029990 con 0x55d002432840 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c15ffb640 1 -- 172.21.15.110:0/734514074 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0c18029b30 con 0x55d002432840 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c377f9740 1 -- 172.21.15.110:0/734514074 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c2809fa40 con 0x55d002432840 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c15ffb640 1 -- 172.21.15.110:0/734514074 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0c18043070 con 0x55d002432840 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c377f9740 10 client.25208 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c377f9740 1 -- 172.21.15.110:0/734514074 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d0024ade00 con 0x55d002432840 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.026+0000 7f0c15ffb640 1 -- 172.21.15.110:0/734514074 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c2809fa40 con 0x55d002432840 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c15ffb640 1 -- 172.21.15.110:0/734514074 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f0c18027050 con 0x55d002432840 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c377f9740 10 client.25208 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c377f9740 10 client.25208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c377f9740 20 client.25208 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c377f9740 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d0024ae780 0x55d0024b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c377f9740 4 client.25208 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c377f9740 1 -- 172.21.15.110:0/734514074 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d00239d540 con 0x55d0024ae780 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c2e59c640 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d0024ae780 0x55d0024b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c15ffb640 10 client.25208 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.027+0000 7f0c2e59c640 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d0024ae780 0x55d0024b0ba0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f5a37c39aa5af3 server_cookie=3f5ef090668c3f92 in_seq=0 out_seq=0 2024-04-25T00:06:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c15ffb640 1 -- 172.21.15.110:0/734514074 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d00239d540 con 0x55d0024ae780 2024-04-25T00:06:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c15ffb640 10 client.25208 handle_command_reply: tid=0 2024-04-25T00:06:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 client.25208 shutdown 2024-04-25T00:06:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 -- 172.21.15.110:0/734514074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 msgr2=0x55d002440a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d002440a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c18029700 tx=0x7f0c18003d30 comp rx=0 tx=0).stop 2024-04-25T00:06:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 -- 172.21.15.110:0/734514074 shutdown_connections 2024-04-25T00:06:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d0024ae780 0x55d0024b0ba0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d0024379b0 0x55d0024ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d002432840 0x55d002440a00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 --2- 172.21.15.110:0/734514074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d002431e50 0x55d0024404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c377f9740 1 -- 172.21.15.110:0/734514074 >> 172.21.15.110:0/734514074 conn(0x55d00242d4c0 msgr2=0x55d00243be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.048+0000 7f0c15ffb640 0 client.25208 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.049+0000 7f0c377f9740 1 -- 172.21.15.110:0/734514074 shutdown_connections 2024-04-25T00:06:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.049+0000 7f0c377f9740 1 -- 172.21.15.110:0/734514074 wait complete. 2024-04-25T00:06:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.049+0000 7f0c377f9740 20 client.25208 trim_cache size 0 max 16384 2024-04-25T00:06:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:06:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:06:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:06:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:06:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c1c0a2cb0 msgr2=0x7f0c1c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c1c0a2cb0 0x7f0c1c0a5170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55d002432c40 tx=0x7f0c18005f90 comp rx=0 tx=0).stop 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c301a36e0 msgr2=0x7f0c301a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c301a36e0 0x7f0c301a5ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800efc0 tx=0x7f0c2800c800 comp rx=0 tx=0).stop 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 shutdown_connections 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c1c0a2cb0 0x7f0c1c0a5170 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c301a36e0 0x7f0c301a5ad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c30077ce0 0x7f0c3019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 --2- 172.21.15.110:0/419821573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c30077340 0x7f0c3019e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 >> 172.21.15.110:0/419821573 conn(0x7f0c300fef60 msgr2=0x7f0c301022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 shutdown_connections 2024-04-25T00:06:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:31.050+0000 7f0c34a12640 1 -- 172.21.15.110:0/419821573 wait complete. 2024-04-25T00:06:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 205.0 s, 824.0 tps, lat 121.260 ms stddev 21.750 2024-04-25T00:06:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:31 smithi188 ceph-mon[33285]: pgmap v485: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.3 MiB/s wr, 956 op/s 2024-04-25T00:06:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:31 smithi110 ceph-mon[29187]: pgmap v485: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.3 MiB/s wr, 956 op/s 2024-04-25T00:06:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:31 smithi173 ceph-mon[32402]: pgmap v485: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.3 MiB/s wr, 956 op/s 2024-04-25T00:06:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:32 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:06:33.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:32 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:06:33.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:32 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:06:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:33 smithi188 ceph-mon[33285]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-04-25T00:06:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:06:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:06:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:06:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:33 smithi110 ceph-mon[29187]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-04-25T00:06:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:06:34.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:06:34.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:06:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:33 smithi173 ceph-mon[32402]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-04-25T00:06:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:06:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:06:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:06:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:35 smithi188 ceph-mon[33285]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.1 MiB/s wr, 976 op/s 2024-04-25T00:06:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:35 smithi110 ceph-mon[29187]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.1 MiB/s wr, 976 op/s 2024-04-25T00:06:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:35 smithi173 ceph-mon[32402]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.1 MiB/s wr, 976 op/s 2024-04-25T00:06:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 210.0 s, 769.4 tps, lat 129.824 ms stddev 25.230 2024-04-25T00:06:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:37 smithi188 ceph-mon[33285]: pgmap v488: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 915 op/s 2024-04-25T00:06:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:37 smithi110 ceph-mon[29187]: pgmap v488: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 915 op/s 2024-04-25T00:06:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:37 smithi173 ceph-mon[32402]: pgmap v488: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 915 op/s 2024-04-25T00:06:40.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:39 smithi188 ceph-mon[33285]: pgmap v489: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-04-25T00:06:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:39 smithi110 ceph-mon[29187]: pgmap v489: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-04-25T00:06:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:39 smithi173 ceph-mon[32402]: pgmap v489: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-04-25T00:06:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 215.0 s, 837.8 tps, lat 119.452 ms stddev 17.806 2024-04-25T00:06:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:41 smithi188 ceph-mon[33285]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 930 op/s 2024-04-25T00:06:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:41 smithi110 ceph-mon[29187]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 930 op/s 2024-04-25T00:06:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:41 smithi173 ceph-mon[32402]: pgmap v490: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 MiB/s wr, 930 op/s 2024-04-25T00:06:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:43 smithi188 ceph-mon[33285]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 1.04k op/s 2024-04-25T00:06:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:43 smithi110 ceph-mon[29187]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 1.04k op/s 2024-04-25T00:06:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:43 smithi173 ceph-mon[32402]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 1.04k op/s 2024-04-25T00:06:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.178+0000 7fbc36b63640 1 -- 172.21.15.110:0/284145268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc30101d50 msgr2=0x7fbc30102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.178+0000 7fbc36b63640 1 --2- 172.21.15.110:0/284145268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc30101d50 0x7fbc30102170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009900 tx=0x7fbc2402a400 comp rx=0 tx=0).stop 2024-04-25T00:06:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.178+0000 7fbc36b63640 1 -- 172.21.15.110:0/284145268 shutdown_connections 2024-04-25T00:06:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.178+0000 7fbc36b63640 1 --2- 172.21.15.110:0/284145268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc301012d0 0x7fbc30108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.178+0000 7fbc36b63640 1 --2- 172.21.15.110:0/284145268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc30100910 0x7fbc30100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.178+0000 7fbc36b63640 1 --2- 172.21.15.110:0/284145268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc30101d50 0x7fbc30102170 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.178+0000 7fbc36b63640 1 -- 172.21.15.110:0/284145268 >> 172.21.15.110:0/284145268 conn(0x7fbc300fb3c0 msgr2=0x7fbc300fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.179+0000 7fbc36b63640 1 -- 172.21.15.110:0/284145268 shutdown_connections 2024-04-25T00:06:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.179+0000 7fbc36b63640 1 -- 172.21.15.110:0/284145268 wait complete. 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.179+0000 7fbc36b63640 1 Processor -- start 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc36b63640 1 -- start start 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc36b63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc30100910 0x7fbc30196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc36b63640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc301012d0 0x7fbc30196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc36b63640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc30101d50 0x7fbc3019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc348ff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc30100910 0x7fbc30196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc36b63640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3010c1f0 con 0x7fbc301012d0 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc36b63640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3010c070 con 0x7fbc30101d50 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc36b63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3010c370 con 0x7fbc30100910 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc301012d0 0x7fbc30196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc301012d0 0x7fbc30196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45500/0 (socket says 172.21.15.110:45500) 2024-04-25T00:06:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc2ffff640 1 -- 172.21.15.110:0/571495706 learned_addr learned my addr 172.21.15.110:0/571495706 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.180+0000 7fbc35100640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc30101d50 0x7fbc3019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2ffff640 1 -- 172.21.15.110:0/571495706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc30100910 msgr2=0x7fbc30196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc30100910 0x7fbc30196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2ffff640 1 -- 172.21.15.110:0/571495706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc30101d50 msgr2=0x7fbc3019dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc30101d50 0x7fbc3019dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2ffff640 1 -- 172.21.15.110:0/571495706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3019e450 con 0x7fbc301012d0 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc301012d0 0x7fbc30196d70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00e9c0 tx=0x7fbc1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2dffb640 1 -- 172.21.15.110:0/571495706 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00db90 con 0x7fbc301012d0 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc36b63640 1 -- 172.21.15.110:0/571495706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc300ff860 con 0x7fbc301012d0 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2dffb640 1 -- 172.21.15.110:0/571495706 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc1c00dd30 con 0x7fbc301012d0 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.181+0000 7fbc2dffb640 1 -- 172.21.15.110:0/571495706 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c003e30 con 0x7fbc301012d0 2024-04-25T00:06:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.182+0000 7fbc36b63640 1 -- 172.21.15.110:0/571495706 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc300ffe00 con 0x7fbc301012d0 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc2dffb640 1 -- 172.21.15.110:0/571495706 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbc1c01d020 con 0x7fbc301012d0 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc39876740 1 Processor -- start 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc39876740 1 -- start start 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc39876740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15742270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc39876740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e15747340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc39876740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e157479b0 0x564e15749e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc39876740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564e156a5cb0 con 0x564e157479b0 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc39876740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564e1564e5b0 con 0x564e15742840 2024-04-25T00:06:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc39876740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564e156a1c70 con 0x564e15741e50 2024-04-25T00:06:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc2dffb640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbc200a2d30 0x7fbc200a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc35100640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15742270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc348ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e15747340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc348ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e15747340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:52290/0 (socket says 172.21.15.110:52290) 2024-04-25T00:06:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc348ff640 1 -- 172.21.15.110:0/507230094 learned_addr learned my addr 172.21.15.110:0/507230094 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.185+0000 7fbc35100640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15742270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38264/0 (socket says 172.21.15.110:38264) 2024-04-25T00:06:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc35100640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbc200a2d30 0x7fbc200a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc2dffb640 1 -- 172.21.15.110:0/571495706 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fbc1c0ce8a0 con 0x7fbc301012d0 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc348ff640 1 -- 172.21.15.110:0/507230094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 msgr2=0x564e15742270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc348ff640 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15742270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc348ff640 1 -- 172.21.15.110:0/507230094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e157479b0 msgr2=0x564e15749e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e157479b0 0x564e15749e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc348ff640 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e157479b0 0x564e15749e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc348ff640 1 -- 172.21.15.110:0/507230094 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e153ba210 con 0x564e15742840 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc35100640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbc200a2d30 0x7fbc200a51f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005c40 tx=0x7fbc28009bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.186+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e157479b0 0x564e15749e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.187+0000 7fbc348ff640 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e15747340 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009900 tx=0x7fbc24005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e91e4e9de5140d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.187+0000 7fbc1affd640 1 -- 172.21.15.110:0/507230094 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24049070 con 0x564e15742840 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.187+0000 7fbc1affd640 1 -- 172.21.15.110:0/507230094 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc24031d70 con 0x564e15742840 2024-04-25T00:06:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.187+0000 7fbc1affd640 1 -- 172.21.15.110:0/507230094 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24035050 con 0x564e15742840 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 -- 172.21.15.110:0/507230094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 msgr2=0x564e15747340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e15747340 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009900 tx=0x7fbc24005c00 comp rx=0 tx=0).stop 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 -- 172.21.15.110:0/507230094 shutdown_connections 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e157479b0 0x564e15749e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e15747340 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 --2- 172.21.15.110:0/507230094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15742270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 -- 172.21.15.110:0/507230094 >> 172.21.15.110:0/507230094 conn(0x564e1573d4c0 msgr2=0x564e1573d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 -- 172.21.15.110:0/507230094 shutdown_connections 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.188+0000 7fbc39876740 1 -- 172.21.15.110:0/507230094 wait complete. 2024-04-25T00:06:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc39876740 1 -- start start 2024-04-25T00:06:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc39876740 1 Processor -- start 2024-04-25T00:06:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc39876740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15741e50 0x564e1574c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc39876740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e1574c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc39876740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e157479b0 0x564e15753830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc348ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15741e50 0x564e1574c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc348ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15741e50 0x564e1574c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:52306/0 (socket says 172.21.15.110:52306) 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc348ff640 1 -- 172.21.15.110:0/199020939 learned_addr learned my addr 172.21.15.110:0/199020939 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e1574c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564e1574a900 con 0x564e15742840 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564e15661b50 con 0x564e15741e50 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564e1574aa80 con 0x564e157479b0 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.189+0000 7fbc35100640 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e157479b0 0x564e15753830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc39876740 10 client.0 _ll_register_callbacks cb 0x564e1569cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc348ff640 1 -- 172.21.15.110:0/199020939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e157479b0 msgr2=0x564e15753830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc348ff640 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e157479b0 0x564e15753830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc348ff640 1 -- 172.21.15.110:0/199020939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 msgr2=0x564e1574c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc348ff640 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e1574c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc348ff640 1 -- 172.21.15.110:0/199020939 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e15486620 con 0x564e15741e50 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e1574c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc24031d70 con 0x564e15741e50 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc348ff640 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15741e50 0x564e1574c2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc24006010 tx=0x7fbc2403aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da7ef910224e02c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.190+0000 7fbc0ffff640 10 client.25220 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc0ffff640 1 -- 172.21.15.110:0/199020939 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24049070 con 0x564e15741e50 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc0ffff640 1 -- 172.21.15.110:0/199020939 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fbc24031b70 con 0x564e15741e50 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc0ffff640 1 -- 172.21.15.110:0/199020939 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbc24009520 con 0x564e15741e50 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc39876740 10 client.25220 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({fsmap=31}) -- 0x564e15756a10 con 0x564e15741e50 2024-04-25T00:06:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc0ffff640 1 -- 172.21.15.110:0/199020939 <== mon.2 v2:172.21.15.173:3300/0 4 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7fbc2404e400 con 0x564e15741e50 2024-04-25T00:06:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc39876740 10 client.25220 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc39876740 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc39876740 20 client.25220 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc39876740 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564e15757d60 0x564e1575a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc39876740 4 client.25220 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e156b31f0 con 0x564e15757d60 2024-04-25T00:06:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.191+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564e15757d60 0x564e1575a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.192+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564e15757d60 0x564e1575a150 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d37179fa3051a77 server_cookie=b900510c23939e12 in_seq=0 out_seq=0 2024-04-25T00:06:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.192+0000 7fbc0ffff640 10 client.25220 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.192+0000 7fbc0ffff640 1 -- 172.21.15.110:0/199020939 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e156b31f0 con 0x564e15757d60 2024-04-25T00:06:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc0ffff640 10 client.25220 handle_command_reply: tid=0 2024-04-25T00:06:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 client.25220 shutdown 2024-04-25T00:06:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15741e50 msgr2=0x564e1574c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15741e50 0x564e1574c2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc24006010 tx=0x7fbc2403aa00 comp rx=0 tx=0).stop 2024-04-25T00:06:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 shutdown_connections 2024-04-25T00:06:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564e15757d60 0x564e1575a150 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e157479b0 0x564e15753830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc0ffff640 0 client.25220 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e1574c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 --2- 172.21.15.110:0/199020939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15741e50 0x564e1574c2b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 >> 172.21.15.110:0/199020939 conn(0x564e1573d4c0 msgr2=0x564e1573c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 shutdown_connections 2024-04-25T00:06:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 1 -- 172.21.15.110:0/199020939 wait complete. 2024-04-25T00:06:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.193+0000 7fbc39876740 20 client.25220 trim_cache size 0 max 16384 2024-04-25T00:06:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.217+0000 7fbc39876740 1 Processor -- start 2024-04-25T00:06:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.217+0000 7fbc39876740 1 -- start start 2024-04-25T00:06:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.217+0000 7fbc39876740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e1573d4c0 0x564e157409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.217+0000 7fbc39876740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15741e50 0x564e1574b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.218+0000 7fbc39876740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e1574be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.218+0000 7fbc39876740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301a9100 con 0x564e15741e50 2024-04-25T00:06:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.218+0000 7fbc39876740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301a69a0 con 0x564e15742840 2024-04-25T00:06:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.218+0000 7fbc39876740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301a47f0 con 0x564e1573d4c0 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.218+0000 7fbc35100640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e1574be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.218+0000 7fbc35100640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e1574be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:52310/0 (socket says 172.21.15.110:52310) 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.218+0000 7fbc35100640 1 -- 172.21.15.110:0/2631617546 learned_addr learned my addr 172.21.15.110:0/2631617546 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.218+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15741e50 0x564e1574b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.219+0000 7fbc35100640 1 -- 172.21.15.110:0/2631617546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e1573d4c0 msgr2=0x564e157409f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.219+0000 7fbc35100640 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e1573d4c0 0x564e157409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.219+0000 7fbc35100640 1 -- 172.21.15.110:0/2631617546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15741e50 msgr2=0x564e1574b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.219+0000 7fbc35100640 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15741e50 0x564e1574b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.219+0000 7fbc35100640 1 -- 172.21.15.110:0/2631617546 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e15746fc0 con 0x564e15742840 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.219+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15741e50 0x564e1574b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.219+0000 7fbc35100640 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e1574be10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009d50 tx=0x7fbc28016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f773f6585e984f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.220+0000 7fbc1a7fc640 1 -- 172.21.15.110:0/2631617546 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28003e00 con 0x564e15742840 2024-04-25T00:06:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.220+0000 7fbc1a7fc640 1 -- 172.21.15.110:0/2631617546 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc28004b20 con 0x564e15742840 2024-04-25T00:06:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.220+0000 7fbc1a7fc640 1 -- 172.21.15.110:0/2631617546 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28005740 con 0x564e15742840 2024-04-25T00:06:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.222+0000 7fbc39876740 1 -- 172.21.15.110:0/2631617546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 msgr2=0x564e1574be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.222+0000 7fbc39876740 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e1574be10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009d50 tx=0x7fbc28016000 comp rx=0 tx=0).stop 2024-04-25T00:06:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.222+0000 7fbc39876740 1 -- 172.21.15.110:0/2631617546 shutdown_connections 2024-04-25T00:06:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.222+0000 7fbc39876740 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e15742840 0x564e1574be10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.222+0000 7fbc39876740 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15741e50 0x564e1574b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.222+0000 7fbc39876740 1 --2- 172.21.15.110:0/2631617546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e1573d4c0 0x564e157409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.222+0000 7fbc39876740 1 -- 172.21.15.110:0/2631617546 >> 172.21.15.110:0/2631617546 conn(0x564e15743620 msgr2=0x564e157402e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.223+0000 7fbc39876740 1 -- 172.21.15.110:0/2631617546 shutdown_connections 2024-04-25T00:06:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.223+0000 7fbc39876740 1 -- 172.21.15.110:0/2631617546 wait complete. 2024-04-25T00:06:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.223+0000 7fbc39876740 1 -- start start 2024-04-25T00:06:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.223+0000 7fbc39876740 1 Processor -- start 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc39876740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e1573d4c0 0x564e157502d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc39876740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15750810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc39876740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e15751820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc39876740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301a4e60 con 0x564e15742840 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc39876740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301a62d0 con 0x564e1573d4c0 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc39876740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301a5530 con 0x564e15741e50 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc39876740 10 client.0 _ll_register_callbacks cb 0x7fbc24047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15750810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15750810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38282/0 (socket says 172.21.15.110:38282) 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc2ffff640 1 -- 172.21.15.110:0/918637961 learned_addr learned my addr 172.21.15.110:0/918637961 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.224+0000 7fbc348ff640 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e1573d4c0 0x564e157502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.225+0000 7fbc35100640 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e15751820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.225+0000 7fbc2ffff640 1 -- 172.21.15.110:0/918637961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e1573d4c0 msgr2=0x564e157502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.225+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e1573d4c0 0x564e157502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.225+0000 7fbc2ffff640 1 -- 172.21.15.110:0/918637961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 msgr2=0x564e15751820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.225+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e15751820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.225+0000 7fbc2ffff640 1 -- 172.21.15.110:0/918637961 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e15751e10 con 0x564e15741e50 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.225+0000 7fbc2ffff640 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15750810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c097530 tx=0x7fbc1c0bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1119b7b31a7005ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.225+0000 7fbc0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.226+0000 7fbc0ffff640 1 -- 172.21.15.110:0/918637961 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c09f480 con 0x564e15741e50 2024-04-25T00:06:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.226+0000 7fbc39876740 1 -- 172.21.15.110:0/918637961 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc28004b20 con 0x564e15741e50 2024-04-25T00:06:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.226+0000 7fbc0ffff640 1 -- 172.21.15.110:0/918637961 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fbc1c0bb0b0 con 0x564e15741e50 2024-04-25T00:06:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.226+0000 7fbc0ffff640 1 -- 172.21.15.110:0/918637961 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbc1c07e050 con 0x564e15741e50 2024-04-25T00:06:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.226+0000 7fbc39876740 10 client.25201 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.226+0000 7fbc39876740 1 -- 172.21.15.110:0/918637961 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564e157c6890 con 0x564e15741e50 2024-04-25T00:06:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.226+0000 7fbc0ffff640 1 -- 172.21.15.110:0/918637961 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc28004b20 con 0x564e15741e50 2024-04-25T00:06:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.226+0000 7fbc0ffff640 1 -- 172.21.15.110:0/918637961 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7fbc1c0a38b0 con 0x564e15741e50 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.227+0000 7fbc39876740 10 client.25201 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.227+0000 7fbc39876740 10 client.25201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.227+0000 7fbc39876740 20 client.25201 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.227+0000 7fbc39876740 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564e157479b0 0x564e157c9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.227+0000 7fbc39876740 4 client.25201 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.227+0000 7fbc39876740 1 -- 172.21.15.110:0/918637961 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e156ad540 con 0x564e157479b0 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.228+0000 7fbc348ff640 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564e157479b0 0x564e157c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.228+0000 7fbc0ffff640 10 client.25201 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.228+0000 7fbc348ff640 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564e157479b0 0x564e157c9170 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e95d8b58563082 server_cookie=b651020a2778bebd in_seq=0 out_seq=0 2024-04-25T00:06:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.247+0000 7fbc0ffff640 1 -- 172.21.15.110:0/918637961 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e156ad540 con 0x564e157479b0 2024-04-25T00:06:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.247+0000 7fbc0ffff640 10 client.25201 handle_command_reply: tid=0 2024-04-25T00:06:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 client.25201 shutdown 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 -- 172.21.15.110:0/918637961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 msgr2=0x564e15750810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15750810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c097530 tx=0x7fbc1c0bb040 comp rx=0 tx=0).stop 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 -- 172.21.15.110:0/918637961 shutdown_connections 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564e157479b0 0x564e157c9170 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e15742840 0x564e15751820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e15741e50 0x564e15750810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 --2- 172.21.15.110:0/918637961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564e1573d4c0 0x564e157502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc39876740 1 -- 172.21.15.110:0/918637961 >> 172.21.15.110:0/918637961 conn(0x564e15743620 msgr2=0x564e15740b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.248+0000 7fbc0ffff640 0 client.25201 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.249+0000 7fbc39876740 1 -- 172.21.15.110:0/918637961 shutdown_connections 2024-04-25T00:06:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.249+0000 7fbc39876740 1 -- 172.21.15.110:0/918637961 wait complete. 2024-04-25T00:06:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.249+0000 7fbc39876740 20 client.25201 trim_cache size 0 max 16384 2024-04-25T00:06:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:06:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:06:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:06:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:06:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 -- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbc200a2d30 msgr2=0x7fbc200a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbc200a2d30 0x7fbc200a51f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005c40 tx=0x7fbc28009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 -- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc301012d0 msgr2=0x7fbc30196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc301012d0 0x7fbc30196d70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00e9c0 tx=0x7fbc1c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:06:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 -- 172.21.15.110:0/571495706 shutdown_connections 2024-04-25T00:06:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fbc200a2d30 0x7fbc200a51f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc30101d50 0x7fbc3019dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc301012d0 0x7fbc30196d70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 --2- 172.21.15.110:0/571495706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc30100910 0x7fbc30196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 -- 172.21.15.110:0/571495706 >> 172.21.15.110:0/571495706 conn(0x7fbc300fb3c0 msgr2=0x7fbc30104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 -- 172.21.15.110:0/571495706 shutdown_connections 2024-04-25T00:06:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:44.250+0000 7fbc36b63640 1 -- 172.21.15.110:0/571495706 wait complete. 2024-04-25T00:06:46.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:45 smithi188 ceph-mon[33285]: pgmap v492: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 984 op/s 2024-04-25T00:06:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:45 smithi110 ceph-mon[29187]: pgmap v492: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 984 op/s 2024-04-25T00:06:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:45 smithi173 ceph-mon[32402]: pgmap v492: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 984 op/s 2024-04-25T00:06:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 220.0 s, 848.0 tps, lat 117.931 ms stddev 12.982 2024-04-25T00:06:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:47 smithi188 ceph-mon[33285]: pgmap v493: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 925 op/s 2024-04-25T00:06:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:47 smithi110 ceph-mon[29187]: pgmap v493: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 925 op/s 2024-04-25T00:06:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:47 smithi173 ceph-mon[32402]: pgmap v493: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 925 op/s 2024-04-25T00:06:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:49 smithi188 ceph-mon[33285]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 1.16k op/s 2024-04-25T00:06:50.039 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:49 smithi110 ceph-mon[29187]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 1.16k op/s 2024-04-25T00:06:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:49 smithi173 ceph-mon[32402]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 1.16k op/s 2024-04-25T00:06:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 225.0 s, 835.2 tps, lat 119.509 ms stddev 16.007 2024-04-25T00:06:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:51 smithi188 ceph-mon[33285]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 942 op/s 2024-04-25T00:06:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:51 smithi110 ceph-mon[29187]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 942 op/s 2024-04-25T00:06:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:51 smithi173 ceph-mon[32402]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 942 op/s 2024-04-25T00:06:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:52 smithi173 ceph-mon[32402]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:52 smithi188 ceph-mon[33285]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:52 smithi110 ceph-mon[29187]: pgmap v496: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 1.03k op/s 2024-04-25T00:06:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:55 smithi188 ceph-mon[33285]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 MiB/s wr, 968 op/s 2024-04-25T00:06:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:55 smithi110 ceph-mon[29187]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 MiB/s wr, 968 op/s 2024-04-25T00:06:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:55 smithi173 ceph-mon[32402]: pgmap v497: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 MiB/s wr, 968 op/s 2024-04-25T00:06:56.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 230.0 s, 834.0 tps, lat 120.277 ms stddev 17.394 2024-04-25T00:06:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.383+0000 7f26b15ce640 1 -- 172.21.15.110:0/1920574507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac100970 msgr2=0x7f26ac100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.383+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1920574507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac100970 0x7f26ac100dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2694009900 tx=0x7f269402a400 comp rx=0 tx=0).stop 2024-04-25T00:06:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.383+0000 7f26b15ce640 1 -- 172.21.15.110:0/1920574507 shutdown_connections 2024-04-25T00:06:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.383+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1920574507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26ac101310 0x7f26ac101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.383+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1920574507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac100970 0x7f26ac100dd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.383+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1920574507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26ac100010 0x7f26ac100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.383+0000 7f26b15ce640 1 -- 172.21.15.110:0/1920574507 >> 172.21.15.110:0/1920574507 conn(0x7f26ac0fbc10 msgr2=0x7f26ac0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.384+0000 7f26b15ce640 1 -- 172.21.15.110:0/1920574507 shutdown_connections 2024-04-25T00:06:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.384+0000 7f26b15ce640 1 -- 172.21.15.110:0/1920574507 wait complete. 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.384+0000 7f26b15ce640 1 Processor -- start 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.384+0000 7f26b15ce640 1 -- start start 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26b15ce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26ac100010 0x7f26ac199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26b15ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26ac100970 0x7f26ac19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26b15ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac101310 0x7f26ac1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26b15ce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac10f4e0 con 0x7f26ac100970 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26b15ce640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac10f360 con 0x7f26ac100010 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26b15ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac10f660 con 0x7f26ac101310 2024-04-25T00:06:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26aaffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26ac100010 0x7f26ac199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26ac100970 0x7f26ac19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26aaffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26ac100010 0x7f26ac199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39452/0 (socket says 172.21.15.110:39452) 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26aaffd640 1 -- 172.21.15.110:0/1396060773 learned_addr learned my addr 172.21.15.110:0/1396060773 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac101310 0x7f26ac1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26aaffd640 1 -- 172.21.15.110:0/1396060773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac101310 msgr2=0x7f26ac1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26aaffd640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac101310 0x7f26ac1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26aaffd640 1 -- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26ac100970 msgr2=0x7f26ac19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26aaffd640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26ac100970 0x7f26ac19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.385+0000 7f26aaffd640 1 -- 172.21.15.110:0/1396060773 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac1a1740 con 0x7f26ac100010 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.386+0000 7f26aaffd640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26ac100010 0x7f26ac199b20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f269c00e9f0 tx=0x7f269c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.386+0000 7f26a8ff9640 1 -- 172.21.15.110:0/1396060773 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c00cd80 con 0x7f26ac100010 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.386+0000 7f26b15ce640 1 -- 172.21.15.110:0/1396060773 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ac1a1a30 con 0x7f26ac100010 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.386+0000 7f26a8ff9640 1 -- 172.21.15.110:0/1396060773 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f269c0108e0 con 0x7f26ac100010 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.386+0000 7f26a8ff9640 1 -- 172.21.15.110:0/1396060773 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c018920 con 0x7f26ac100010 2024-04-25T00:06:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.386+0000 7f26b15ce640 1 -- 172.21.15.110:0/1396060773 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ac1a1fa0 con 0x7f26ac100010 2024-04-25T00:06:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.389+0000 7f26a8ff9640 1 -- 172.21.15.110:0/1396060773 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f269c010a90 con 0x7f26ac100010 2024-04-25T00:06:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.389+0000 7f26a8ff9640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f26980a2c80 0x7f26980a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.389+0000 7f26a8ff9640 1 -- 172.21.15.110:0/1396060773 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f269c0cf2e0 con 0x7f26ac100010 2024-04-25T00:06:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.390+0000 7f26a3fff640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f26980a2c80 0x7f26980a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.390+0000 7f26a3fff640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f26980a2c80 0x7f26980a5140 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2694005ab0 tx=0x7f2694000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26b43bc740 1 Processor -- start 2024-04-25T00:06:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26b43bc740 1 -- start start 2024-04-25T00:06:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb6275660 0x562eb6275a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb627ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb627b1c0 0x562eb627d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26b43bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562eb61d0cb0 con 0x562eb627b1c0 2024-04-25T00:06:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26b43bc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562eb61795b0 con 0x562eb6275660 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26aaffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb6275660 0x562eb6275a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26ab7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb627b1c0 0x562eb627d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26aaffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb6275660 0x562eb6275a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39456/0 (socket says 172.21.15.110:39456) 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26aaffd640 1 -- 172.21.15.110:0/1235868184 learned_addr learned my addr 172.21.15.110:0/1235868184 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.391+0000 7f26b43bc740 1 -- 172.21.15.110:0/1235868184 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562eb61ccc70 con 0x562eb6276050 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26a3fff640 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb627ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26ab7fe640 1 -- 172.21.15.110:0/1235868184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 msgr2=0x562eb627ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb627ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26ab7fe640 1 -- 172.21.15.110:0/1235868184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb6275660 msgr2=0x562eb6275a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb6275660 0x562eb6275a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26ab7fe640 1 -- 172.21.15.110:0/1235868184 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb5ee5210 con 0x562eb627b1c0 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26aaffd640 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb6275660 0x562eb6275a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26a3fff640 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb627ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb627b1c0 0x562eb627d650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009f30 tx=0x7f26a4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dcb67aaf30d8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.392+0000 7f26a17fa640 1 -- 172.21.15.110:0/1235868184 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4016070 con 0x562eb627b1c0 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.393+0000 7f26a17fa640 1 -- 172.21.15.110:0/1235868184 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a4004d20 con 0x562eb627b1c0 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.393+0000 7f26a17fa640 1 -- 172.21.15.110:0/1235868184 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4012740 con 0x562eb627b1c0 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 -- 172.21.15.110:0/1235868184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb627b1c0 msgr2=0x562eb627d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb627b1c0 0x562eb627d650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009f30 tx=0x7f26a4010800 comp rx=0 tx=0).stop 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 -- 172.21.15.110:0/1235868184 shutdown_connections 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb627b1c0 0x562eb627d650 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb627ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 --2- 172.21.15.110:0/1235868184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb6275660 0x562eb6275a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 -- 172.21.15.110:0/1235868184 >> 172.21.15.110:0/1235868184 conn(0x562eb62684c0 msgr2=0x562eb62688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 -- 172.21.15.110:0/1235868184 shutdown_connections 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.394+0000 7f26b43bc740 1 -- 172.21.15.110:0/1235868184 wait complete. 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26b43bc740 1 -- start start 2024-04-25T00:06:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26b43bc740 1 Processor -- start 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6275660 0x562eb627f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6276050 0x562eb627fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26aaffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6275660 0x562eb627f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26aaffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6275660 0x562eb627f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48930/0 (socket says 172.21.15.110:48930) 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6276050 0x562eb627fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26aaffd640 1 -- 172.21.15.110:0/3305066288 learned_addr learned my addr 172.21.15.110:0/3305066288 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.395+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6276050 0x562eb627fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39094/0 (socket says 172.21.15.110:39094) 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.396+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.396+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562eb627e170 con 0x562eb6276050 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.396+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562eb618cb50 con 0x562eb627b1c0 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.396+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562eb627e2f0 con 0x562eb6275660 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.396+0000 7f26b43bc740 10 client.0 _ll_register_callbacks cb 0x562eb61c7ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26aaffd640 1 -- 172.21.15.110:0/3305066288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 msgr2=0x562eb6286e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26aaffd640 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26aaffd640 1 -- 172.21.15.110:0/3305066288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6276050 msgr2=0x562eb627fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26aaffd640 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6276050 0x562eb627fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26aaffd640 1 -- 172.21.15.110:0/3305066288 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb5fb1620 con 0x562eb6275660 2024-04-25T00:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26a3fff640 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6276050 0x562eb627fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a4004d20 con 0x562eb6275660 2024-04-25T00:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26aaffd640 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6275660 0x562eb627f910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f269c00e9c0 tx=0x7f269c09bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce4b0e2789f3b203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26927fc640 10 client.25210 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26927fc640 1 -- 172.21.15.110:0/3305066288 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c0ba4b0 con 0x562eb6275660 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.397+0000 7f26927fc640 1 -- 172.21.15.110:0/3305066288 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f269c010430 con 0x562eb6275660 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26927fc640 1 -- 172.21.15.110:0/3305066288 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f269c010630 con 0x562eb6275660 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26b43bc740 10 client.25210 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=31}) -- 0x562eb628a130 con 0x562eb6275660 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26927fc640 1 -- 172.21.15.110:0/3305066288 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f269c09b070 con 0x562eb6275660 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26b43bc740 10 client.25210 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26b43bc740 10 client.25210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26b43bc740 20 client.25210 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562eb628b760 0x562eb628dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26b43bc740 4 client.25210 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eb61de1f0 con 0x562eb628b760 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.398+0000 7f26a3fff640 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562eb628b760 0x562eb628dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.399+0000 7f26a3fff640 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562eb628b760 0x562eb628dbf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7bb2cbda40396b3 server_cookie=1d83c2587489d702 in_seq=0 out_seq=0 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.399+0000 7f26927fc640 10 client.25210 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.399+0000 7f26927fc640 1 -- 172.21.15.110:0/3305066288 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562eb61de1f0 con 0x562eb628b760 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.399+0000 7f26927fc640 10 client.25210 handle_command_reply: tid=0 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.399+0000 7f26b43bc740 1 client.25210 shutdown 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6275660 msgr2=0x562eb627f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6275660 0x562eb627f910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f269c00e9c0 tx=0x7f269c09bce0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 shutdown_connections 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562eb628b760 0x562eb628dbf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6276050 0x562eb627fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3305066288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6275660 0x562eb627f910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 >> 172.21.15.110:0/3305066288 conn(0x562eb62684c0 msgr2=0x562eb62675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26927fc640 0 client.25210 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 shutdown_connections 2024-04-25T00:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 1 -- 172.21.15.110:0/3305066288 wait complete. 2024-04-25T00:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.400+0000 7f26b43bc740 20 client.25210 trim_cache size 0 max 16384 2024-04-25T00:06:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.425+0000 7f26b43bc740 1 Processor -- start 2024-04-25T00:06:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.425+0000 7f26b43bc740 1 -- start start 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb62934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb62939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26aaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb62934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26aaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb62934a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39110/0 (socket says 172.21.15.110:39110) 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26aaffd640 1 -- 172.21.15.110:0/281202730 learned_addr learned my addr 172.21.15.110:0/281202730 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26b43bc740 1 -- 172.21.15.110:0/281202730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac1a91f0 con 0x562eb6275660 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26b43bc740 1 -- 172.21.15.110:0/281202730 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac1a6a90 con 0x562eb627b1c0 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26b43bc740 1 -- 172.21.15.110:0/281202730 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac1a48e0 con 0x562eb6276050 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.426+0000 7f26a3fff640 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb62939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26ab7fe640 1 -- 172.21.15.110:0/281202730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 msgr2=0x562eb62939e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb62939e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26ab7fe640 1 -- 172.21.15.110:0/281202730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 msgr2=0x562eb62934a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb62934a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26ab7fe640 1 -- 172.21.15.110:0/281202730 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb6269230 con 0x562eb627b1c0 2024-04-25T00:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26aaffd640 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb62934a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005a00 tx=0x7f26a4010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=983679b4e00934ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26a0ff9640 1 -- 172.21.15.110:0/281202730 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4016040 con 0x562eb627b1c0 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26a0ff9640 1 -- 172.21.15.110:0/281202730 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a4003b70 con 0x562eb627b1c0 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.427+0000 7f26a0ff9640 1 -- 172.21.15.110:0/281202730 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4005130 con 0x562eb627b1c0 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.428+0000 7f26b43bc740 1 -- 172.21.15.110:0/281202730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 msgr2=0x562eb6286e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.428+0000 7f26b43bc740 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005a00 tx=0x7f26a4010fd0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.428+0000 7f26b43bc740 1 -- 172.21.15.110:0/281202730 shutdown_connections 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.428+0000 7f26b43bc740 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6286e50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.428+0000 7f26b43bc740 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb62939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 --2- 172.21.15.110:0/281202730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb62934a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 -- 172.21.15.110:0/281202730 >> 172.21.15.110:0/281202730 conn(0x562eb62684c0 msgr2=0x562eb62688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 -- 172.21.15.110:0/281202730 shutdown_connections 2024-04-25T00:06:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 -- 172.21.15.110:0/281202730 wait complete. 2024-04-25T00:06:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 -- start start 2024-04-25T00:06:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 Processor -- start 2024-04-25T00:06:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb6284130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb6284670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6285680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac1a4f50 con 0x562eb6275660 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac1a63c0 con 0x562eb627b1c0 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac1a5620 con 0x562eb6276050 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.429+0000 7f26b43bc740 10 client.0 _ll_register_callbacks cb 0x7f269c010de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb6284670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb6284670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48944/0 (socket says 172.21.15.110:48944) 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 -- 172.21.15.110:0/3539413010 learned_addr learned my addr 172.21.15.110:0/3539413010 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26ab7fe640 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6285680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26aaffd640 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb6284130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 -- 172.21.15.110:0/3539413010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 msgr2=0x562eb6285680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6285680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 -- 172.21.15.110:0/3539413010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 msgr2=0x562eb6284130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb6284130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 -- 172.21.15.110:0/3539413010 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb6285c70 con 0x562eb6276050 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26a3fff640 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb6284670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f269402a9f0 tx=0x7f269402a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f834183de77a8ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26927fc640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.430+0000 7f26927fc640 1 -- 172.21.15.110:0/3539413010 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694057070 con 0x562eb6276050 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.431+0000 7f26927fc640 1 -- 172.21.15.110:0/3539413010 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2694052440 con 0x562eb6276050 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.431+0000 7f26b43bc740 1 -- 172.21.15.110:0/3539413010 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a4003b70 con 0x562eb6276050 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.431+0000 7f26927fc640 1 -- 172.21.15.110:0/3539413010 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2694035050 con 0x562eb6276050 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.431+0000 7f26b43bc740 10 client.25216 fetch_fsmap learned FSMap version 31 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.431+0000 7f26b43bc740 1 -- 172.21.15.110:0/3539413010 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562eb6286ff0 con 0x562eb6276050 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.431+0000 7f26927fc640 1 -- 172.21.15.110:0/3539413010 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a4003b70 con 0x562eb6276050 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26927fc640 1 -- 172.21.15.110:0/3539413010 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f269403ca10 con 0x562eb6276050 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26b43bc740 10 client.25216 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26b43bc740 10 client.25216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26b43bc740 20 client.25216 populate_metadata read hostname 'smithi110' 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562eb62f2350 0x562eb62f4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26b43bc740 4 client.25216 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:06:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26b43bc740 1 -- 172.21.15.110:0/3539413010 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562eb61d8540 con 0x562eb62f2350 2024-04-25T00:06:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26aaffd640 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562eb62f2350 0x562eb62f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26927fc640 10 client.25216 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.432+0000 7f26aaffd640 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562eb62f2350 0x562eb62f4770 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a1897db40e97e2 server_cookie=ef2fb5678da22de6 in_seq=0 out_seq=0 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26927fc640 1 -- 172.21.15.110:0/3539413010 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562eb61d8540 con 0x562eb62f2350 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26927fc640 10 client.25216 handle_command_reply: tid=0 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 client.25216 shutdown 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 -- 172.21.15.110:0/3539413010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 msgr2=0x562eb6284670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb6284670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f269402a9f0 tx=0x7f269402a400 comp rx=0 tx=0).stop 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 -- 172.21.15.110:0/3539413010 shutdown_connections 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562eb62f2350 0x562eb62f4770 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562eb627b1c0 0x562eb6285680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562eb6276050 0x562eb6284670 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 --2- 172.21.15.110:0/3539413010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562eb6275660 0x562eb6284130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 -- 172.21.15.110:0/3539413010 >> 172.21.15.110:0/3539413010 conn(0x562eb62684c0 msgr2=0x562eb626b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26927fc640 0 client.25216 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 -- 172.21.15.110:0/3539413010 shutdown_connections 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.454+0000 7f26b43bc740 1 -- 172.21.15.110:0/3539413010 wait complete. 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.455+0000 7f26b43bc740 20 client.25216 trim_cache size 0 max 16384 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:06:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 -- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f26980a2c80 msgr2=0x7f26980a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f26980a2c80 0x7f26980a5140 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2694005ab0 tx=0x7f2694000bc0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 -- 172.21.15.110:0/1396060773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26ac100010 msgr2=0x7f26ac199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26ac100010 0x7f26ac199b20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f269c00e9f0 tx=0x7f269c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 -- 172.21.15.110:0/1396060773 shutdown_connections 2024-04-25T00:06:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f26980a2c80 0x7f26980a5140 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26ac101310 0x7f26ac1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26ac100970 0x7f26ac19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 --2- 172.21.15.110:0/1396060773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26ac100010 0x7f26ac199b20 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f269c00e9f0 tx=0x7f269c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.456+0000 7f26b15ce640 1 -- 172.21.15.110:0/1396060773 >> 172.21.15.110:0/1396060773 conn(0x7f26ac0fbc10 msgr2=0x7f26ac0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.457+0000 7f26b15ce640 1 -- 172.21.15.110:0/1396060773 shutdown_connections 2024-04-25T00:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:06:57.457+0000 7f26b15ce640 1 -- 172.21.15.110:0/1396060773 wait complete. 2024-04-25T00:06:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:57 smithi188 ceph-mon[33285]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 862 op/s 2024-04-25T00:06:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:57 smithi110 ceph-mon[29187]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 862 op/s 2024-04-25T00:06:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:57 smithi173 ceph-mon[32402]: pgmap v498: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 MiB/s wr, 862 op/s 2024-04-25T00:07:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:06:59 smithi188 ceph-mon[33285]: pgmap v499: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 1.08k op/s 2024-04-25T00:07:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:06:59 smithi110 ceph-mon[29187]: pgmap v499: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 1.08k op/s 2024-04-25T00:07:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:06:59 smithi173 ceph-mon[32402]: pgmap v499: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 1.08k op/s 2024-04-25T00:07:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 235.0 s, 824.8 tps, lat 121.091 ms stddev 15.446 2024-04-25T00:07:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:01 smithi188 ceph-mon[33285]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.5 MiB/s wr, 895 op/s 2024-04-25T00:07:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:01 smithi110 ceph-mon[29187]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.5 MiB/s wr, 895 op/s 2024-04-25T00:07:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:01 smithi173 ceph-mon[32402]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.5 MiB/s wr, 895 op/s 2024-04-25T00:07:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:03 smithi188 ceph-mon[33285]: pgmap v501: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 897 op/s 2024-04-25T00:07:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:03 smithi110 ceph-mon[29187]: pgmap v501: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 897 op/s 2024-04-25T00:07:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:03 smithi173 ceph-mon[32402]: pgmap v501: 97 pgs: 97 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 897 op/s 2024-04-25T00:07:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:05 smithi188 ceph-mon[33285]: pgmap v502: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 988 op/s 2024-04-25T00:07:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:05 smithi110 ceph-mon[29187]: pgmap v502: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 988 op/s 2024-04-25T00:07:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:05 smithi173 ceph-mon[32402]: pgmap v502: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 988 op/s 2024-04-25T00:07:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 240.0 s, 825.8 tps, lat 120.891 ms stddev 17.185 2024-04-25T00:07:08.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:07 smithi188 ceph-mon[33285]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 925 op/s 2024-04-25T00:07:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:07 smithi110 ceph-mon[29187]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 925 op/s 2024-04-25T00:07:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:07 smithi173 ceph-mon[32402]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 925 op/s 2024-04-25T00:07:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:09 smithi188 ceph-mon[33285]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 1.14k op/s 2024-04-25T00:07:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:09 smithi110 ceph-mon[29187]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 1.14k op/s 2024-04-25T00:07:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:09 smithi173 ceph-mon[32402]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 1.14k op/s 2024-04-25T00:07:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 -- 172.21.15.110:0/2038576908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 msgr2=0x7fa284100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2038576908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 0x7fa284100d20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa27800b9f0 tx=0x7fa27802c0a0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 -- 172.21.15.110:0/2038576908 shutdown_connections 2024-04-25T00:07:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2038576908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa284101260 0x7fa284108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2038576908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 0x7fa284100d20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2038576908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa284101c70 0x7fa284102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 -- 172.21.15.110:0/2038576908 >> 172.21.15.110:0/2038576908 conn(0x7fa2840fb3b0 msgr2=0x7fa2840fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 -- 172.21.15.110:0/2038576908 shutdown_connections 2024-04-25T00:07:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.572+0000 7fa28a87d640 1 -- 172.21.15.110:0/2038576908 wait complete. 2024-04-25T00:07:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa28a87d640 1 Processor -- start 2024-04-25T00:07:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa28a87d640 1 -- start start 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa28a87d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 0x7fa2841956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa28a87d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa284101260 0x7fa284195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa28a87d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa284101c70 0x7fa28419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa28a87d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410b0f0 con 0x7fa2841008c0 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa28a87d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410af70 con 0x7fa284101c70 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa28a87d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410b270 con 0x7fa284101260 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa283fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 0x7fa2841956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa283fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 0x7fa2841956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39168/0 (socket says 172.21.15.110:39168) 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa283fff640 1 -- 172.21.15.110:0/2264768001 learned_addr learned my addr 172.21.15.110:0/2264768001 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa2837fe640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa284101260 0x7fa284195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.573+0000 7fa288e1a640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa284101c70 0x7fa28419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2837fe640 1 -- 172.21.15.110:0/2264768001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa284101c70 msgr2=0x7fa28419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2837fe640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa284101c70 0x7fa28419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2837fe640 1 -- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 msgr2=0x7fa2841956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2837fe640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 0x7fa2841956f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2837fe640 1 -- 172.21.15.110:0/2264768001 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28419d310 con 0x7fa284101260 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa283fff640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 0x7fa2841956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2837fe640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa284101260 0x7fa284195c30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa278009540 tx=0x7fa278009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2817fa640 1 -- 172.21.15.110:0/2264768001 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278041030 con 0x7fa284101260 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa28a87d640 1 -- 172.21.15.110:0/2264768001 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28419d5a0 con 0x7fa284101260 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2817fa640 1 -- 172.21.15.110:0/2264768001 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa278004200 con 0x7fa284101260 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa2817fa640 1 -- 172.21.15.110:0/2264768001 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278039040 con 0x7fa284101260 2024-04-25T00:07:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.574+0000 7fa28a87d640 1 -- 172.21.15.110:0/2264768001 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28419dae0 con 0x7fa284101260 2024-04-25T00:07:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.577+0000 7fa2817fa640 1 -- 172.21.15.110:0/2264768001 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa278044070 con 0x7fa284101260 2024-04-25T00:07:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.577+0000 7fa28d58d740 1 Processor -- start 2024-04-25T00:07:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.577+0000 7fa28d58d740 1 -- start start 2024-04-25T00:07:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff15f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff15fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15fddc0 0x556ff1600250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa28d58d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556ff1557cb0 con 0x556ff15fddc0 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa28d58d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ff15005b0 con 0x556ff15f8c50 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa28d58d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556ff1553c70 con 0x556ff15f8260 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa2817fa640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa26c0a2cb0 0x7fa26c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa2837fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15fddc0 0x556ff1600250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa283fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff15fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa288e1a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff15f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa283fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff15fd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:42554/0 (socket says 172.21.15.110:42554) 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa283fff640 1 -- 172.21.15.110:0/1327238050 learned_addr learned my addr 172.21.15.110:0/1327238050 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa2837fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15fddc0 0x556ff1600250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39182/0 (socket says 172.21.15.110:39182) 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa288e1a640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa26c0a2cb0 0x7fa26c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa283fff640 1 -- 172.21.15.110:0/1327238050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 msgr2=0x556ff15f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa283fff640 1 --2- 172.21.15.110:0/1327238050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff15f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa283fff640 1 -- 172.21.15.110:0/1327238050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15fddc0 msgr2=0x556ff1600250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa283fff640 1 --2- 172.21.15.110:0/1327238050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15fddc0 0x556ff1600250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa283fff640 1 -- 172.21.15.110:0/1327238050 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff126c210 con 0x556ff15f8c50 2024-04-25T00:07:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.578+0000 7fa2817fa640 1 -- 172.21.15.110:0/2264768001 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa2780f0880 con 0x7fa284101260 2024-04-25T00:07:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.579+0000 7fa283fff640 1 --2- 172.21.15.110:0/1327238050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff15fd750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa274011650 tx=0x7fa274009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8968a46e44a1ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.579+0000 7fa288e1a640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa26c0a2cb0 0x7fa26c0a5170 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x556ff15f94b0 tx=0x7fa27c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.579+0000 7fa26a7fc640 1 -- 172.21.15.110:0/1327238050 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27401e070 con 0x556ff15f8c50 2024-04-25T00:07:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.579+0000 7fa26a7fc640 1 -- 172.21.15.110:0/1327238050 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa27400fa90 con 0x556ff15f8c50 2024-04-25T00:07:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.579+0000 7fa26a7fc640 1 -- 172.21.15.110:0/1327238050 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274019b80 con 0x556ff15f8c50 2024-04-25T00:07:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 -- 172.21.15.110:0/1327238050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 msgr2=0x556ff15fd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 --2- 172.21.15.110:0/1327238050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff15fd750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa274011650 tx=0x7fa274009f90 comp rx=0 tx=0).stop 2024-04-25T00:07:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 -- 172.21.15.110:0/1327238050 shutdown_connections 2024-04-25T00:07:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 --2- 172.21.15.110:0/1327238050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15fddc0 0x556ff1600250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 --2- 172.21.15.110:0/1327238050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff15fd750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 --2- 172.21.15.110:0/1327238050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff15f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 -- 172.21.15.110:0/1327238050 >> 172.21.15.110:0/1327238050 conn(0x556ff15ef4c0 msgr2=0x556ff15ef8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 -- 172.21.15.110:0/1327238050 shutdown_connections 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 -- 172.21.15.110:0/1327238050 wait complete. 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.580+0000 7fa28d58d740 1 -- start start 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa28d58d740 1 Processor -- start 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1602720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15f8c50 0x556ff1602c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15fddc0 0x556ff1609ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa28d58d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556ff1600d70 con 0x556ff15f8c50 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa28d58d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ff1513b50 con 0x556ff15fddc0 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa28d58d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556ff1600ef0 con 0x556ff15f8260 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa28d58d740 10 client.0 _ll_register_callbacks cb 0x556ff154eca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa2837fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15f8c50 0x556ff1602c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa283fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1602720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa283fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1602720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:41904/0 (socket says 172.21.15.110:41904) 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa283fff640 1 -- 172.21.15.110:0/2336674986 learned_addr learned my addr 172.21.15.110:0/2336674986 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa288e1a640 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15fddc0 0x556ff1609ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa283fff640 1 -- 172.21.15.110:0/2336674986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15fddc0 msgr2=0x556ff1609ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa283fff640 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15fddc0 0x556ff1609ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa283fff640 1 -- 172.21.15.110:0/2336674986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15f8c50 msgr2=0x556ff1602c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa283fff640 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15f8c50 0x556ff1602c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.581+0000 7fa283fff640 1 -- 172.21.15.110:0/2336674986 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff1338620 con 0x556ff15f8260 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa283fff640 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1602720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa274019450 tx=0x7fa274005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f99a6ef0ad6d36d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa2637fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa2637fe640 1 -- 172.21.15.110:0/2336674986 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274027060 con 0x556ff15f8260 2024-04-25T00:07:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa2637fe640 1 -- 172.21.15.110:0/2336674986 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa27400f6e0 con 0x556ff15f8260 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa28d58d740 1 -- 172.21.15.110:0/2336674986 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27400fa90 con 0x556ff15f8260 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa2637fe640 1 -- 172.21.15.110:0/2336674986 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa27401e040 con 0x556ff15f8260 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa28d58d740 10 client.25228 fetch_fsmap learned FSMap version 31 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa28d58d740 1 -- 172.21.15.110:0/2336674986 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556ff160d060 con 0x556ff15f8260 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.582+0000 7fa2637fe640 1 -- 172.21.15.110:0/2336674986 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa27400fa90 con 0x556ff15f8260 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa2637fe640 1 -- 172.21.15.110:0/2336674986 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7fa274021070 con 0x556ff15f8260 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa28d58d740 10 client.25228 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa28d58d740 10 client.25228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa28d58d740 20 client.25228 populate_metadata read hostname 'smithi110' 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa28d58d740 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556ff160e4b0 0x556ff16108a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa28d58d740 4 client.25228 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa28d58d740 1 -- 172.21.15.110:0/2336674986 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ff15651f0 con 0x556ff160e4b0 2024-04-25T00:07:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa2837fe640 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556ff160e4b0 0x556ff16108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa2637fe640 10 client.25228 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.583+0000 7fa2837fe640 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556ff160e4b0 0x556ff16108a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4940e79ae348958b server_cookie=52fc06a8744381ed in_seq=0 out_seq=0 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.584+0000 7fa2637fe640 1 -- 172.21.15.110:0/2336674986 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ff15651f0 con 0x556ff160e4b0 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.584+0000 7fa2637fe640 10 client.25228 handle_command_reply: tid=0 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.584+0000 7fa28d58d740 1 client.25228 shutdown 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.584+0000 7fa28d58d740 1 -- 172.21.15.110:0/2336674986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 msgr2=0x556ff1602720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.584+0000 7fa28d58d740 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1602720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa274019450 tx=0x7fa274005d30 comp rx=0 tx=0).stop 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.584+0000 7fa28d58d740 1 -- 172.21.15.110:0/2336674986 shutdown_connections 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.584+0000 7fa28d58d740 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556ff160e4b0 0x556ff16108a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.585+0000 7fa28d58d740 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15fddc0 0x556ff1609ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.585+0000 7fa28d58d740 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15f8c50 0x556ff1602c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.585+0000 7fa2637fe640 0 client.25228 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.585+0000 7fa28d58d740 1 --2- 172.21.15.110:0/2336674986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1602720 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.585+0000 7fa28d58d740 1 -- 172.21.15.110:0/2336674986 >> 172.21.15.110:0/2336674986 conn(0x556ff15ef4c0 msgr2=0x556ff15ee5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.585+0000 7fa28d58d740 1 -- 172.21.15.110:0/2336674986 shutdown_connections 2024-04-25T00:07:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.585+0000 7fa28d58d740 1 -- 172.21.15.110:0/2336674986 wait complete. 2024-04-25T00:07:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.585+0000 7fa28d58d740 20 client.25228 trim_cache size 0 max 16384 2024-04-25T00:07:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa28d58d740 1 Processor -- start 2024-04-25T00:07:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa28d58d740 1 -- start start 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff15f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8260 0x556ff1609c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8c50 0x556ff15ee440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa28d58d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841a9080 con 0x556ff15ef4c0 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa28d58d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841a6920 con 0x556ff15f8260 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa28d58d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841a4770 con 0x556ff15f8c50 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa283fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff15f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa283fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff15f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39202/0 (socket says 172.21.15.110:39202) 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa2837fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8260 0x556ff1609c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa283fff640 1 -- 172.21.15.110:0/178520855 learned_addr learned my addr 172.21.15.110:0/178520855 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa288e1a640 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8c50 0x556ff15ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa2837fe640 1 -- 172.21.15.110:0/178520855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8c50 msgr2=0x556ff15ee440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa2837fe640 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8c50 0x556ff15ee440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa2837fe640 1 -- 172.21.15.110:0/178520855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 msgr2=0x556ff15f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa2837fe640 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff15f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa2837fe640 1 -- 172.21.15.110:0/178520855 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff15f0230 con 0x556ff15f8260 2024-04-25T00:07:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.606+0000 7fa283fff640 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff15f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.607+0000 7fa2837fe640 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8260 0x556ff1609c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa2780ee400 tx=0x7fa2780dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a8c390e6dc68f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.607+0000 7fa269ffb640 1 -- 172.21.15.110:0/178520855 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2780bcc50 con 0x556ff15f8260 2024-04-25T00:07:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.607+0000 7fa269ffb640 1 -- 172.21.15.110:0/178520855 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2780bd810 con 0x556ff15f8260 2024-04-25T00:07:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.607+0000 7fa269ffb640 1 -- 172.21.15.110:0/178520855 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2780043b0 con 0x556ff15f8260 2024-04-25T00:07:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 -- 172.21.15.110:0/178520855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8260 msgr2=0x556ff1609c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8260 0x556ff1609c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa2780ee400 tx=0x7fa2780dd040 comp rx=0 tx=0).stop 2024-04-25T00:07:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 -- 172.21.15.110:0/178520855 shutdown_connections 2024-04-25T00:07:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8c50 0x556ff15ee440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8260 0x556ff1609c60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 --2- 172.21.15.110:0/178520855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff15f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 -- 172.21.15.110:0/178520855 >> 172.21.15.110:0/178520855 conn(0x556ff15f9a30 msgr2=0x556ff15edd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 -- 172.21.15.110:0/178520855 shutdown_connections 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 -- 172.21.15.110:0/178520855 wait complete. 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.608+0000 7fa28d58d740 1 -- start start 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa28d58d740 1 Processor -- start 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff16077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1607d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa28d58d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff1608cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa28d58d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841a4de0 con 0x556ff15ef4c0 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa28d58d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841a6250 con 0x556ff15f8c50 2024-04-25T00:07:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa28d58d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841a54b0 con 0x556ff15f8260 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa28d58d740 10 client.0 _ll_register_callbacks cb 0x7fa27401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa288e1a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff1608cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa288e1a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff1608cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:42578/0 (socket says 172.21.15.110:42578) 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa288e1a640 1 -- 172.21.15.110:0/3398034897 learned_addr learned my addr 172.21.15.110:0/3398034897 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa283fff640 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff16077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa2837fe640 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1607d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa288e1a640 1 -- 172.21.15.110:0/3398034897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 msgr2=0x556ff1607d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa288e1a640 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1607d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa288e1a640 1 -- 172.21.15.110:0/3398034897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 msgr2=0x556ff16077f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa288e1a640 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff16077f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa288e1a640 1 -- 172.21.15.110:0/3398034897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff16092e0 con 0x556ff15f8c50 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.609+0000 7fa2837fe640 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1607d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa288e1a640 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff1608cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004700 tx=0x7fa27c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa84b73134267b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa2637fe640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa2637fe640 1 -- 172.21.15.110:0/3398034897 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c030060 con 0x556ff15f8c50 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa2637fe640 1 -- 172.21.15.110:0/3398034897 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa27c01fb90 con 0x556ff15f8c50 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa28d58d740 1 -- 172.21.15.110:0/3398034897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2780bd810 con 0x556ff15f8c50 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa2637fe640 1 -- 172.21.15.110:0/3398034897 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa27c01fd90 con 0x556ff15f8c50 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa28d58d740 10 client.25262 fetch_fsmap learned FSMap version 31 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa28d58d740 1 -- 172.21.15.110:0/3398034897 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556ff1674e20 con 0x556ff15f8c50 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa2637fe640 1 -- 172.21.15.110:0/3398034897 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2780bd810 con 0x556ff15f8c50 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.610+0000 7fa2637fe640 1 -- 172.21.15.110:0/3398034897 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7fa27c00b040 con 0x556ff15f8c50 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa28d58d740 10 client.25262 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa28d58d740 10 client.25262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa28d58d740 20 client.25262 populate_metadata read hostname 'smithi110' 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa28d58d740 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556ff15fddc0 0x556ff16778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa28d58d740 4 client.25262 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:07:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa28d58d740 1 -- 172.21.15.110:0/3398034897 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ff155f540 con 0x556ff15fddc0 2024-04-25T00:07:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa283fff640 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556ff15fddc0 0x556ff16778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa2637fe640 10 client.25262 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.611+0000 7fa283fff640 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556ff15fddc0 0x556ff16778c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a53c5e6ba7e2f7f server_cookie=c997e30c7f1a6663 in_seq=0 out_seq=0 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa2637fe640 1 -- 172.21.15.110:0/3398034897 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ff155f540 con 0x556ff15fddc0 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa2637fe640 10 client.25262 handle_command_reply: tid=0 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 client.25262 shutdown 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 -- 172.21.15.110:0/3398034897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 msgr2=0x556ff1608cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff1608cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004700 tx=0x7fa27c004a40 comp rx=0 tx=0).stop 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 -- 172.21.15.110:0/3398034897 shutdown_connections 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556ff15fddc0 0x556ff16778c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ff15f8c50 0x556ff1608cf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556ff15f8260 0x556ff1607d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 --2- 172.21.15.110:0/3398034897 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ff15ef4c0 0x556ff16077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa2637fe640 0 client.25262 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.627+0000 7fa28d58d740 1 -- 172.21.15.110:0/3398034897 >> 172.21.15.110:0/3398034897 conn(0x556ff15f9a30 msgr2=0x556ff15f2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.628+0000 7fa28d58d740 1 -- 172.21.15.110:0/3398034897 shutdown_connections 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.628+0000 7fa28d58d740 1 -- 172.21.15.110:0/3398034897 wait complete. 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.628+0000 7fa28d58d740 20 client.25262 trim_cache size 0 max 16384 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:07:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:07:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.628+0000 7fa28a87d640 1 -- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa26c0a2cb0 msgr2=0x7fa26c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.628+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa26c0a2cb0 0x7fa26c0a5170 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x556ff15f94b0 tx=0x7fa27c004ab0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.628+0000 7fa28a87d640 1 -- 172.21.15.110:0/2264768001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa284101260 msgr2=0x7fa284195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.628+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa284101260 0x7fa284195c30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa278009540 tx=0x7fa278009ad0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.629+0000 7fa28a87d640 1 -- 172.21.15.110:0/2264768001 shutdown_connections 2024-04-25T00:07:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.629+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa26c0a2cb0 0x7fa26c0a5170 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.629+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa284101c70 0x7fa28419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.629+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa284101260 0x7fa284195c30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.629+0000 7fa28a87d640 1 --2- 172.21.15.110:0/2264768001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa2841008c0 0x7fa2841956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.629+0000 7fa28a87d640 1 -- 172.21.15.110:0/2264768001 >> 172.21.15.110:0/2264768001 conn(0x7fa2840fb3b0 msgr2=0x7fa284104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.629+0000 7fa28a87d640 1 -- 172.21.15.110:0/2264768001 shutdown_connections 2024-04-25T00:07:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:10.629+0000 7fa28a87d640 1 -- 172.21.15.110:0/2264768001 wait complete. 2024-04-25T00:07:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 245.0 s, 789.8 tps, lat 126.972 ms stddev 78.059 2024-04-25T00:07:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:11 smithi188 ceph-mon[33285]: pgmap v505: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 MiB/s wr, 938 op/s 2024-04-25T00:07:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:11 smithi110 ceph-mon[29187]: pgmap v505: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 MiB/s wr, 938 op/s 2024-04-25T00:07:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:11 smithi173 ceph-mon[32402]: pgmap v505: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 MiB/s wr, 938 op/s 2024-04-25T00:07:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:13 smithi188 ceph-mon[33285]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.8 MiB/s wr, 1.09k op/s 2024-04-25T00:07:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:13 smithi110 ceph-mon[29187]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.8 MiB/s wr, 1.09k op/s 2024-04-25T00:07:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:13 smithi173 ceph-mon[32402]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.8 MiB/s wr, 1.09k op/s 2024-04-25T00:07:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:15 smithi188 ceph-mon[33285]: pgmap v507: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-04-25T00:07:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:15 smithi110 ceph-mon[29187]: pgmap v507: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-04-25T00:07:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:15 smithi173 ceph-mon[32402]: pgmap v507: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-04-25T00:07:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 250.0 s, 900.4 tps, lat 110.982 ms stddev 10.979 2024-04-25T00:07:18.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:17 smithi188 ceph-mon[33285]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 MiB/s wr, 851 op/s 2024-04-25T00:07:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:17 smithi110 ceph-mon[29187]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 MiB/s wr, 851 op/s 2024-04-25T00:07:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:17 smithi173 ceph-mon[32402]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.4 MiB/s wr, 851 op/s 2024-04-25T00:07:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:19 smithi188 ceph-mon[33285]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-04-25T00:07:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:19 smithi110 ceph-mon[29187]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-04-25T00:07:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:19 smithi173 ceph-mon[32402]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-04-25T00:07:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 255.0 s, 899.4 tps, lat 110.977 ms stddev 15.930 2024-04-25T00:07:22.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:21 smithi188 ceph-mon[33285]: pgmap v510: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 910 op/s 2024-04-25T00:07:22.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:21 smithi110 ceph-mon[29187]: pgmap v510: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 910 op/s 2024-04-25T00:07:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:21 smithi173 ceph-mon[32402]: pgmap v510: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 910 op/s 2024-04-25T00:07:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 -- 172.21.15.110:0/766293864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c1009e0 msgr2=0x7ff02c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 --2- 172.21.15.110:0/766293864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c1009e0 0x7ff02c100e40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff020009860 tx=0x7ff02002a340 comp rx=0 tx=0).stop 2024-04-25T00:07:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 -- 172.21.15.110:0/766293864 shutdown_connections 2024-04-25T00:07:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 --2- 172.21.15.110:0/766293864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c101380 0x7ff02c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 --2- 172.21.15.110:0/766293864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c1009e0 0x7ff02c100e40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 --2- 172.21.15.110:0/766293864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c100080 0x7ff02c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 -- 172.21.15.110:0/766293864 >> 172.21.15.110:0/766293864 conn(0x7ff02c0fbc40 msgr2=0x7ff02c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 -- 172.21.15.110:0/766293864 shutdown_connections 2024-04-25T00:07:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.765+0000 7ff030d63640 1 -- 172.21.15.110:0/766293864 wait complete. 2024-04-25T00:07:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff030d63640 1 Processor -- start 2024-04-25T00:07:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff030d63640 1 -- start start 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff030d63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c100080 0x7ff02c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff030d63640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c1009e0 0x7ff02c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff030d63640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c101380 0x7ff02c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff030d63640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c10f4a0 con 0x7ff02c1009e0 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff030d63640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c10f320 con 0x7ff02c101380 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff030d63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c10f620 con 0x7ff02c100080 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.766+0000 7ff029d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c1009e0 0x7ff02c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff029d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c1009e0 0x7ff02c19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45274/0 (socket says 172.21.15.110:45274) 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff029d9b640 1 -- 172.21.15.110:0/3109006042 learned_addr learned my addr 172.21.15.110:0/3109006042 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c101380 0x7ff02c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff02a59c640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c100080 0x7ff02c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff02ad9d640 1 -- 172.21.15.110:0/3109006042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c100080 msgr2=0x7ff02c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c100080 0x7ff02c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff02ad9d640 1 -- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c1009e0 msgr2=0x7ff02c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c1009e0 0x7ff02c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff02ad9d640 1 -- 172.21.15.110:0/3109006042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c1a1760 con 0x7ff02c101380 2024-04-25T00:07:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.767+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c101380 0x7ff02c1a10f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff02400ec50 tx=0x7ff02400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.768+0000 7ff01b7fe640 1 -- 172.21.15.110:0/3109006042 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024003b70 con 0x7ff02c101380 2024-04-25T00:07:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.768+0000 7ff01b7fe640 1 -- 172.21.15.110:0/3109006042 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02400ee90 con 0x7ff02c101380 2024-04-25T00:07:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.768+0000 7ff030d63640 1 -- 172.21.15.110:0/3109006042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c1a1a50 con 0x7ff02c101380 2024-04-25T00:07:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.768+0000 7ff01b7fe640 1 -- 172.21.15.110:0/3109006042 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024010670 con 0x7ff02c101380 2024-04-25T00:07:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.768+0000 7ff030d63640 1 -- 172.21.15.110:0/3109006042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a1f50 con 0x7ff02c101380 2024-04-25T00:07:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.770+0000 7ff01b7fe640 1 -- 172.21.15.110:0/3109006042 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff024004020 con 0x7ff02c101380 2024-04-25T00:07:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.771+0000 7ff01b7fe640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff0140a2c30 0x7ff0140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.771+0000 7ff01b7fe640 1 -- 172.21.15.110:0/3109006042 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff0240cf590 con 0x7ff02c101380 2024-04-25T00:07:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.771+0000 7ff033a7c740 1 Processor -- start 2024-04-25T00:07:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff02a59c640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff0140a2c30 0x7ff0140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff033a7c740 1 -- start start 2024-04-25T00:07:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377db270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c377e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff033a7c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562c3773ecb0 con 0x562c377dae50 2024-04-25T00:07:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff033a7c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562c376e75b0 con 0x562c377db840 2024-04-25T00:07:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff033a7c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562c3773ac70 con 0x562c377e09b0 2024-04-25T00:07:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff029d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff029d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:42204/0 (socket says 172.21.15.110:42204) 2024-04-25T00:07:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff029d9b640 1 -- 172.21.15.110:0/1284753737 learned_addr learned my addr 172.21.15.110:0/1284753737 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.772+0000 7ff02a59c640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff0140a2c30 0x7ff0140a50f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005fd0 tx=0x7ff01c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02a59c640 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377db270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c377e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02ad9d640 1 -- 172.21.15.110:0/1284753737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 msgr2=0x562c377e0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02ad9d640 1 -- 172.21.15.110:0/1284753737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 msgr2=0x562c377db270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377db270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02ad9d640 1 -- 172.21.15.110:0/1284753737 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c37453210 con 0x562c377e09b0 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02a59c640 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377db270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.773+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c377e2e40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff02409bd50 tx=0x7ff02409b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bdfca5253397410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.774+0000 7ff018ff9640 1 -- 172.21.15.110:0/1284753737 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024003e10 con 0x562c377e09b0 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.774+0000 7ff018ff9640 1 -- 172.21.15.110:0/1284753737 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02409c4c0 con 0x562c377e09b0 2024-04-25T00:07:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.774+0000 7ff018ff9640 1 -- 172.21.15.110:0/1284753737 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02409a190 con 0x562c377e09b0 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 -- 172.21.15.110:0/1284753737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 msgr2=0x562c377e2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c377e2e40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff02409bd50 tx=0x7ff02409b900 comp rx=0 tx=0).stop 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 -- 172.21.15.110:0/1284753737 shutdown_connections 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c377e2e40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 --2- 172.21.15.110:0/1284753737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377db270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 -- 172.21.15.110:0/1284753737 >> 172.21.15.110:0/1284753737 conn(0x562c377d64c0 msgr2=0x562c377d68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 -- 172.21.15.110:0/1284753737 shutdown_connections 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 -- 172.21.15.110:0/1284753737 wait complete. 2024-04-25T00:07:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 -- start start 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.775+0000 7ff033a7c740 1 Processor -- start 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377ec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff033a7c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562c377e38f0 con 0x562c377dae50 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff033a7c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562c376fab50 con 0x562c377e09b0 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff033a7c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562c377e3a70 con 0x562c377db840 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff033a7c740 10 client.0 _ll_register_callbacks cb 0x562c37735ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff02ad9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff02a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff02ad9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377ec780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:42216/0 (socket says 172.21.15.110:42216) 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff02ad9d640 1 -- 172.21.15.110:0/210283171 learned_addr learned my addr 172.21.15.110:0/210283171 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff029d9b640 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff029d9b640 1 -- 172.21.15.110:0/210283171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 msgr2=0x562c377ec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff029d9b640 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377ec780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff029d9b640 1 -- 172.21.15.110:0/210283171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 msgr2=0x562c377e5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff029d9b640 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff029d9b640 1 -- 172.21.15.110:0/210283171 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3751f620 con 0x562c377db840 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.776+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377ec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff029d9b640 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377e5740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x562c377d7430 tx=0x7ff020005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5909041759dfedd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff011ffb640 1 -- 172.21.15.110:0/210283171 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020037040 con 0x562c377db840 2024-04-25T00:07:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff033a7c740 1 -- 172.21.15.110:0/210283171 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff02409c4c0 con 0x562c377db840 2024-04-25T00:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff011ffb640 1 -- 172.21.15.110:0/210283171 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff020004450 con 0x562c377db840 2024-04-25T00:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff011ffb640 1 -- 172.21.15.110:0/210283171 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff020030900 con 0x562c377db840 2024-04-25T00:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff033a7c740 10 client.25246 fetch_fsmap learned FSMap version 31 2024-04-25T00:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff033a7c740 1 -- 172.21.15.110:0/210283171 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c377efbb0 con 0x562c377db840 2024-04-25T00:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff011ffb640 1 -- 172.21.15.110:0/210283171 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02409c4c0 con 0x562c377db840 2024-04-25T00:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff011ffb640 1 -- 172.21.15.110:0/210283171 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7ff020032740 con 0x562c377db840 2024-04-25T00:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff033a7c740 10 client.25246 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:07:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff033a7c740 10 client.25246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:07:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.777+0000 7ff033a7c740 20 client.25246 populate_metadata read hostname 'smithi110' 2024-04-25T00:07:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.778+0000 7ff033a7c740 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562c377f1700 0x562c377f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.778+0000 7ff033a7c740 4 client.25246 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.778+0000 7ff033a7c740 1 -- 172.21.15.110:0/210283171 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c3774c1f0 con 0x562c377f1700 2024-04-25T00:07:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.778+0000 7ff02a59c640 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562c377f1700 0x562c377f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.778+0000 7ff02a59c640 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562c377f1700 0x562c377f3b60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea843db8f5d3452b server_cookie=7a334f2a94596760 in_seq=0 out_seq=0 2024-04-25T00:07:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.778+0000 7ff011ffb640 10 client.25246 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.779+0000 7ff011ffb640 1 -- 172.21.15.110:0/210283171 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c3774c1f0 con 0x562c377f1700 2024-04-25T00:07:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.779+0000 7ff011ffb640 10 client.25246 handle_command_reply: tid=0 2024-04-25T00:07:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.779+0000 7ff033a7c740 1 client.25246 shutdown 2024-04-25T00:07:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 -- 172.21.15.110:0/210283171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 msgr2=0x562c377e5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377e5740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x562c377d7430 tx=0x7ff020005b20 comp rx=0 tx=0).stop 2024-04-25T00:07:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 -- 172.21.15.110:0/210283171 shutdown_connections 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562c377f1700 0x562c377f3b60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377ec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377e5740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 --2- 172.21.15.110:0/210283171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 -- 172.21.15.110:0/210283171 >> 172.21.15.110:0/210283171 conn(0x562c377d64c0 msgr2=0x562c377d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff011ffb640 0 client.25246 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 -- 172.21.15.110:0/210283171 shutdown_connections 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 1 -- 172.21.15.110:0/210283171 wait complete. 2024-04-25T00:07:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.780+0000 7ff033a7c740 20 client.25246 trim_cache size 0 max 16384 2024-04-25T00:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff033a7c740 1 Processor -- start 2024-04-25T00:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff033a7c740 1 -- start start 2024-04-25T00:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377dcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377ec740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff033a7c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1ad540 con 0x562c377dae50 2024-04-25T00:07:23.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff033a7c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1aade0 con 0x562c377e09b0 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff033a7c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1a8c30 con 0x562c377db840 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff029d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff02a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377dcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff029d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377ec740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37286/0 (socket says 172.21.15.110:37286) 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.804+0000 7ff029d9b640 1 -- 172.21.15.110:0/2905070966 learned_addr learned my addr 172.21.15.110:0/2905070966 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff029d9b640 1 -- 172.21.15.110:0/2905070966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 msgr2=0x562c377dd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff029d9b640 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377dd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff029d9b640 1 -- 172.21.15.110:0/2905070966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 msgr2=0x562c377dcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff029d9b640 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377dcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff029d9b640 1 -- 172.21.15.110:0/2905070966 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c377e3bb0 con 0x562c377db840 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff02a59c640 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377dcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:23.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff029d9b640 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377ec740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cef1a2e3785c976 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff013fff640 1 -- 172.21.15.110:0/2905070966 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020032e10 con 0x562c377db840 2024-04-25T00:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff013fff640 1 -- 172.21.15.110:0/2905070966 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff020038040 con 0x562c377db840 2024-04-25T00:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.805+0000 7ff013fff640 1 -- 172.21.15.110:0/2905070966 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200326c0 con 0x562c377db840 2024-04-25T00:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.806+0000 7ff033a7c740 1 -- 172.21.15.110:0/2905070966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 msgr2=0x562c377ec740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.806+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377ec740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020009550 comp rx=0 tx=0).stop 2024-04-25T00:07:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.806+0000 7ff033a7c740 1 -- 172.21.15.110:0/2905070966 shutdown_connections 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.806+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377e09b0 0x562c377dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.806+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377db840 0x562c377ec740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.806+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2905070966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377dcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.806+0000 7ff033a7c740 1 -- 172.21.15.110:0/2905070966 >> 172.21.15.110:0/2905070966 conn(0x562c377d64c0 msgr2=0x562c377e5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.807+0000 7ff033a7c740 1 -- 172.21.15.110:0/2905070966 shutdown_connections 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.807+0000 7ff033a7c740 1 -- 172.21.15.110:0/2905070966 wait complete. 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.807+0000 7ff033a7c740 1 -- start start 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.807+0000 7ff033a7c740 1 Processor -- start 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.807+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.807+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff033a7c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c37855cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff02a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff02a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45310/0 (socket says 172.21.15.110:45310) 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff033a7c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1a92a0 con 0x562c377dae50 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:42232/0 (socket says 172.21.15.110:42232) 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff033a7c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1aa710 con 0x562c377db840 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 -- 172.21.15.110:0/2521535074 learned_addr learned my addr 172.21.15.110:0/2521535074 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1a9970 con 0x562c377e09b0 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff033a7c740 10 client.0 _ll_register_callbacks cb 0x7ff01c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff02ad9d640 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c37855cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 -- 172.21.15.110:0/2521535074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 msgr2=0x562c37855cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c37855cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 -- 172.21.15.110:0/2521535074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 msgr2=0x562c377e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 -- 172.21.15.110:0/2521535074 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c377ec8e0 con 0x562c377db840 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff02a59c640 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff029d9b640 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e9810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff020032d90 tx=0x7ff020032910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b353b0f3ad830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.808+0000 7ff011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:07:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.809+0000 7ff011ffb640 1 -- 172.21.15.110:0/2521535074 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020032e10 con 0x562c377db840 2024-04-25T00:07:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.809+0000 7ff011ffb640 1 -- 172.21.15.110:0/2521535074 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff0200035f0 con 0x562c377db840 2024-04-25T00:07:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.809+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff020038040 con 0x562c377db840 2024-04-25T00:07:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.809+0000 7ff011ffb640 1 -- 172.21.15.110:0/2521535074 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff020005850 con 0x562c377db840 2024-04-25T00:07:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.809+0000 7ff033a7c740 10 client.25280 fetch_fsmap learned FSMap version 31 2024-04-25T00:07:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.809+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c37857150 con 0x562c377db840 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.809+0000 7ff011ffb640 1 -- 172.21.15.110:0/2521535074 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff020038040 con 0x562c377db840 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.809+0000 7ff011ffb640 1 -- 172.21.15.110:0/2521535074 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7ff02003d900 con 0x562c377db840 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.810+0000 7ff033a7c740 10 client.25280 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.810+0000 7ff033a7c740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.810+0000 7ff033a7c740 20 client.25280 populate_metadata read hostname 'smithi110' 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.810+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562c378579f0 0x562c37859e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.810+0000 7ff033a7c740 4 client.25280 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.810+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c37746540 con 0x562c378579f0 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.810+0000 7ff02a59c640 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562c378579f0 0x562c37859e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.811+0000 7ff011ffb640 10 client.25280 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:23.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.811+0000 7ff02a59c640 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562c378579f0 0x562c37859e10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21409761aa5d825 server_cookie=585f18bd35278086 in_seq=0 out_seq=0 2024-04-25T00:07:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.840+0000 7ff011ffb640 1 -- 172.21.15.110:0/2521535074 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c37746540 con 0x562c378579f0 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.840+0000 7ff011ffb640 10 client.25280 handle_command_reply: tid=0 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.840+0000 7ff033a7c740 1 client.25280 shutdown 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 msgr2=0x562c377e9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e9810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff020032d90 tx=0x7ff020032910 comp rx=0 tx=0).stop 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 shutdown_connections 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x562c378579f0 0x562c37859e10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff011ffb640 0 client.25280 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c377e09b0 0x562c37855cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c377db840 0x562c377e9810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 --2- 172.21.15.110:0/2521535074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562c377dae50 0x562c377e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 >> 172.21.15.110:0/2521535074 conn(0x562c377d64c0 msgr2=0x562c377e50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 shutdown_connections 2024-04-25T00:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 1 -- 172.21.15.110:0/2521535074 wait complete. 2024-04-25T00:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.841+0000 7ff033a7c740 20 client.25280 trim_cache size 0 max 16384 2024-04-25T00:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:07:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 -- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff0140a2c30 msgr2=0x7ff0140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff0140a2c30 0x7ff0140a50f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005fd0 tx=0x7ff01c009260 comp rx=0 tx=0).stop 2024-04-25T00:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 -- 172.21.15.110:0/3109006042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c101380 msgr2=0x7ff02c1a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c101380 0x7ff02c1a10f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff02400ec50 tx=0x7ff02400c6a0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 -- 172.21.15.110:0/3109006042 shutdown_connections 2024-04-25T00:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff0140a2c30 0x7ff0140a50f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff02c101380 0x7ff02c1a10f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff02c1009e0 0x7ff02c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 --2- 172.21.15.110:0/3109006042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02c100080 0x7ff02c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 -- 172.21.15.110:0/3109006042 >> 172.21.15.110:0/3109006042 conn(0x7ff02c0fbc40 msgr2=0x7ff02c0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 -- 172.21.15.110:0/3109006042 shutdown_connections 2024-04-25T00:07:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:23.842+0000 7ff030d63640 1 -- 172.21.15.110:0/3109006042 wait complete. 2024-04-25T00:07:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:23 smithi188 ceph-mon[33285]: pgmap v511: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:07:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:23 smithi110 ceph-mon[29187]: pgmap v511: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:07:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:23 smithi173 ceph-mon[32402]: pgmap v511: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:07:26.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:25 smithi188 ceph-mon[33285]: pgmap v512: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 994 op/s 2024-04-25T00:07:26.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:25 smithi110 ceph-mon[29187]: pgmap v512: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 994 op/s 2024-04-25T00:07:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:25 smithi173 ceph-mon[32402]: pgmap v512: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 994 op/s 2024-04-25T00:07:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 260.0 s, 769.0 tps, lat 130.072 ms stddev 79.634 2024-04-25T00:07:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:27 smithi188 ceph-mon[33285]: pgmap v513: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 943 op/s 2024-04-25T00:07:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:27 smithi110 ceph-mon[29187]: pgmap v513: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 943 op/s 2024-04-25T00:07:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:27 smithi173 ceph-mon[32402]: pgmap v513: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 943 op/s 2024-04-25T00:07:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:29 smithi188 ceph-mon[33285]: pgmap v514: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-04-25T00:07:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:29 smithi110 ceph-mon[29187]: pgmap v514: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-04-25T00:07:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:29 smithi173 ceph-mon[32402]: pgmap v514: 97 pgs: 97 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-04-25T00:07:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 265.0 s, 738.4 tps, lat 135.725 ms stddev 113.664 2024-04-25T00:07:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:31 smithi188 ceph-mon[33285]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-04-25T00:07:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:31 smithi110 ceph-mon[29187]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-04-25T00:07:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:31 smithi173 ceph-mon[32402]: pgmap v515: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-04-25T00:07:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:33 smithi188 ceph-mon[33285]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-04-25T00:07:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:33 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:33 smithi110 ceph-mon[29187]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-04-25T00:07:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:33 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:33 smithi173 ceph-mon[32402]: pgmap v516: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-04-25T00:07:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:33 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:35.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:35.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:35.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:07:35.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:35.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:35.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:07:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:07:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:35 smithi188 ceph-mon[33285]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 976 op/s 2024-04-25T00:07:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:35 smithi188 ceph-mon[33285]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:07:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:35 smithi110 ceph-mon[29187]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 976 op/s 2024-04-25T00:07:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:35 smithi110 ceph-mon[29187]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:07:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:35 smithi173 ceph-mon[32402]: pgmap v517: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 976 op/s 2024-04-25T00:07:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:35 smithi173 ceph-mon[32402]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:07:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 270.0 s, 726.4 tps, lat 137.463 ms stddev 116.045 2024-04-25T00:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 -- 172.21.15.110:0/69450710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd630104c10 msgr2=0x7fd630107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 --2- 172.21.15.110:0/69450710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd630104c10 0x7fd630107000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd620009900 tx=0x7fd62002a410 comp rx=0 tx=0).stop 2024-04-25T00:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 -- 172.21.15.110:0/69450710 shutdown_connections 2024-04-25T00:07:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 --2- 172.21.15.110:0/69450710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd630107540 0x7fd630109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 --2- 172.21.15.110:0/69450710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd630104c10 0x7fd630107000 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 --2- 172.21.15.110:0/69450710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6301022e0 0x7fd6301046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 -- 172.21.15.110:0/69450710 >> 172.21.15.110:0/69450710 conn(0x7fd6300fbc10 msgr2=0x7fd6300fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 -- 172.21.15.110:0/69450710 shutdown_connections 2024-04-25T00:07:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.972+0000 7fd636996640 1 -- 172.21.15.110:0/69450710 wait complete. 2024-04-25T00:07:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.973+0000 7fd636996640 1 Processor -- start 2024-04-25T00:07:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.973+0000 7fd636996640 1 -- start start 2024-04-25T00:07:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.973+0000 7fd636996640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6301022e0 0x7fd630197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.973+0000 7fd636996640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd630104c10 0x7fd630197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd636996640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd630107540 0x7fd63019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd636996640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd63010d300 con 0x7fd630107540 2024-04-25T00:07:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd636996640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd63010d180 con 0x7fd630104c10 2024-04-25T00:07:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd636996640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd63010d480 con 0x7fd6301022e0 2024-04-25T00:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6301022e0 0x7fd630197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62f7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd630104c10 0x7fd630197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6301022e0 0x7fd630197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37740/0 (socket says 172.21.15.110:37740) 2024-04-25T00:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62ffff640 1 -- 172.21.15.110:0/752947488 learned_addr learned my addr 172.21.15.110:0/752947488 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd634f33640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd630107540 0x7fd63019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62f7fe640 1 -- 172.21.15.110:0/752947488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6301022e0 msgr2=0x7fd630197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6301022e0 0x7fd630197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62f7fe640 1 -- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd630107540 msgr2=0x7fd63019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd630107540 0x7fd63019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62f7fe640 1 -- 172.21.15.110:0/752947488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63019f2e0 con 0x7fd630104c10 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.974+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd630104c10 0x7fd630197c50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd620009230 tx=0x7fd620003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.975+0000 7fd62d7fa640 1 -- 172.21.15.110:0/752947488 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62003f070 con 0x7fd630104c10 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.975+0000 7fd62d7fa640 1 -- 172.21.15.110:0/752947488 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd620031dd0 con 0x7fd630104c10 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.975+0000 7fd636996640 1 -- 172.21.15.110:0/752947488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63019f570 con 0x7fd630104c10 2024-04-25T00:07:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.975+0000 7fd62d7fa640 1 -- 172.21.15.110:0/752947488 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620035030 con 0x7fd630104c10 2024-04-25T00:07:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.975+0000 7fd636996640 1 -- 172.21.15.110:0/752947488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63019f9c0 con 0x7fd630104c10 2024-04-25T00:07:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.977+0000 7fd62d7fa640 1 -- 172.21.15.110:0/752947488 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd620042070 con 0x7fd630104c10 2024-04-25T00:07:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.978+0000 7fd62d7fa640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd61c0a2cb0 0x7fd61c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd639777740 1 Processor -- start 2024-04-25T00:07:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd62d7fa640 1 -- 172.21.15.110:0/752947488 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd6200efa70 con 0x7fd630104c10 2024-04-25T00:07:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd639777740 1 -- start start 2024-04-25T00:07:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd62ffff640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd61c0a2cb0 0x7fd61c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd639777740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4260 0x55b63dad4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd639777740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4c50 0x55b63dad9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd639777740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dadc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd639777740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b63da33cb0 con 0x55b63dad4260 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd639777740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b63d9dc5b0 con 0x55b63dad9dc0 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd639777740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b63da2fc70 con 0x55b63dad4c50 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd62f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4c50 0x55b63dad9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd634f33640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dadc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd62f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4c50 0x55b63dad9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37750/0 (socket says 172.21.15.110:37750) 2024-04-25T00:07:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd62f7fe640 1 -- 172.21.15.110:0/552441009 learned_addr learned my addr 172.21.15.110:0/552441009 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd634f33640 1 -- 172.21.15.110:0/552441009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4c50 msgr2=0x55b63dad9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd634f33640 1 --2- 172.21.15.110:0/552441009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4c50 0x55b63dad9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd634f33640 1 -- 172.21.15.110:0/552441009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4260 msgr2=0x55b63dad4680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd634f33640 1 --2- 172.21.15.110:0/552441009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4260 0x55b63dad4680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd634f33640 1 -- 172.21.15.110:0/552441009 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b63d748210 con 0x55b63dad9dc0 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.979+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/552441009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4c50 0x55b63dad9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.980+0000 7fd634f33640 1 --2- 172.21.15.110:0/552441009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dadc250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd628007e40 tx=0x7fd62800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44717d5adbf4d876 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.980+0000 7fd62ffff640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd61c0a2cb0 0x7fd61c0a5170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd618009c90 tx=0x7fd618005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.980+0000 7fd6267fc640 1 -- 172.21.15.110:0/552441009 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628003b70 con 0x55b63dad9dc0 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.980+0000 7fd6267fc640 1 -- 172.21.15.110:0/552441009 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd628003d10 con 0x55b63dad9dc0 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.980+0000 7fd6267fc640 1 -- 172.21.15.110:0/552441009 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628004450 con 0x55b63dad9dc0 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 -- 172.21.15.110:0/552441009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 msgr2=0x55b63dadc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 --2- 172.21.15.110:0/552441009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dadc250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd628007e40 tx=0x7fd62800f790 comp rx=0 tx=0).stop 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 -- 172.21.15.110:0/552441009 shutdown_connections 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 --2- 172.21.15.110:0/552441009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dadc250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 --2- 172.21.15.110:0/552441009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4c50 0x55b63dad9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 --2- 172.21.15.110:0/552441009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4260 0x55b63dad4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 -- 172.21.15.110:0/552441009 >> 172.21.15.110:0/552441009 conn(0x55b63dacf8d0 msgr2=0x55b63dacfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 -- 172.21.15.110:0/552441009 shutdown_connections 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 -- 172.21.15.110:0/552441009 wait complete. 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.981+0000 7fd639777740 1 -- start start 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd639777740 1 Processor -- start 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd639777740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4260 0x55b63dade7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd639777740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dadece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd639777740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad9dc0 0x55b63dae5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd62f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dadece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4260 0x55b63dade7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd62f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dadece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36722/0 (socket says 172.21.15.110:36722) 2024-04-25T00:07:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd62f7fe640 1 -- 172.21.15.110:0/188943370 learned_addr learned my addr 172.21.15.110:0/188943370 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd634f33640 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad9dc0 0x55b63dae5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd639777740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b63dadcd70 con 0x55b63dad4c50 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b63d9efb50 con 0x55b63dad4260 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b63dadcef0 con 0x55b63dad9dc0 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.982+0000 7fd639777740 10 client.0 _ll_register_callbacks cb 0x55b63da2aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.983+0000 7fd62f7fe640 1 -- 172.21.15.110:0/188943370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad9dc0 msgr2=0x55b63dae5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.983+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad9dc0 0x55b63dae5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.983+0000 7fd62f7fe640 1 -- 172.21.15.110:0/188943370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4260 msgr2=0x55b63dade7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.983+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4260 0x55b63dade7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.983+0000 7fd62f7fe640 1 -- 172.21.15.110:0/188943370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b63d814620 con 0x55b63dad4c50 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.983+0000 7fd634f33640 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad9dc0 0x55b63dae5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.983+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dadece0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55b63dad56c0 tx=0x7fd6200bbd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127f88af1bcf6436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.983+0000 7fd60f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd60f7fe640 1 -- 172.21.15.110:0/188943370 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620049040 con 0x55b63dad4c50 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd60f7fe640 1 -- 172.21.15.110:0/188943370 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd6200b89c0 con 0x55b63dad4c50 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd628003d10 con 0x55b63dad4c50 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd60f7fe640 1 -- 172.21.15.110:0/188943370 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd6200c4f10 con 0x55b63dad4c50 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 10 client.15705 fetch_fsmap learned FSMap version 32 2024-04-25T00:07:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b63dae9020 con 0x55b63dad4c50 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd60f7fe640 1 -- 172.21.15.110:0/188943370 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6200d5070 con 0x55b63dad4c50 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd60f7fe640 1 -- 172.21.15.110:0/188943370 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fd6200c4a80 con 0x55b63dad4c50 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 10 client.15705 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 20 client.15705 populate_metadata read hostname 'smithi110' 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b63daea7f0 0x55b63daecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 4 client.15705 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.984+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd628003d10 con 0x55b63daea7f0 2024-04-25T00:07:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.985+0000 7fd62ffff640 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b63daea7f0 0x55b63daecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.985+0000 7fd62ffff640 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b63daea7f0 0x55b63daecc10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c14b8c8988e5eb0 server_cookie=d0e746283bc42655 in_seq=0 out_seq=0 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.985+0000 7fd60f7fe640 10 client.15705 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.986+0000 7fd60f7fe640 1 -- 172.21.15.110:0/188943370 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fd628003d10 con 0x55b63daea7f0 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.986+0000 7fd60f7fe640 10 client.15705 handle_command_reply: tid=0 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.986+0000 7fd639777740 1 client.15705 shutdown 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 msgr2=0x55b63dadece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dadece0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55b63dad56c0 tx=0x7fd6200bbd70 comp rx=0 tx=0).stop 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 shutdown_connections 2024-04-25T00:07:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b63daea7f0 0x55b63daecc10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad9dc0 0x55b63dae5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dadece0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 --2- 172.21.15.110:0/188943370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4260 0x55b63dade7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 >> 172.21.15.110:0/188943370 conn(0x55b63dacf8d0 msgr2=0x55b63daca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd60f7fe640 0 client.15705 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 shutdown_connections 2024-04-25T00:07:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 1 -- 172.21.15.110:0/188943370 wait complete. 2024-04-25T00:07:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:36.987+0000 7fd639777740 20 client.15705 trim_cache size 0 max 16384 2024-04-25T00:07:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.011+0000 7fd639777740 1 Processor -- start 2024-04-25T00:07:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.011+0000 7fd639777740 1 -- start start 2024-04-25T00:07:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd639777740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63daca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd639777740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4c50 0x55b63dae5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd639777740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad9dc0 0x55b63daf14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd639777740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301a8db0 con 0x55b63dad9dc0 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd639777740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301a6680 con 0x55b63dad4c50 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd639777740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301a44d0 con 0x55b63dad4260 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63daca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62f7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4c50 0x55b63dae5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63daca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37772/0 (socket says 172.21.15.110:37772) 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 -- 172.21.15.110:0/831056554 learned_addr learned my addr 172.21.15.110:0/831056554 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62f7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4c50 0x55b63dae5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:40578/0 (socket says 172.21.15.110:40578) 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd634f33640 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad9dc0 0x55b63daf14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 -- 172.21.15.110:0/831056554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4c50 msgr2=0x55b63dae5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4c50 0x55b63dae5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 -- 172.21.15.110:0/831056554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad9dc0 msgr2=0x55b63daf14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad9dc0 0x55b63daf14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 -- 172.21.15.110:0/831056554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b63dadd860 con 0x55b63dad4260 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4c50 0x55b63dae5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.012+0000 7fd62ffff640 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63daca440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd618011ca0 tx=0x7fd618014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19022032d117c30c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.013+0000 7fd625ffb640 1 -- 172.21.15.110:0/831056554 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61802f070 con 0x55b63dad4260 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.013+0000 7fd625ffb640 1 -- 172.21.15.110:0/831056554 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd61802a670 con 0x55b63dad4260 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.013+0000 7fd625ffb640 1 -- 172.21.15.110:0/831056554 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800e3b0 con 0x55b63dad4260 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 -- 172.21.15.110:0/831056554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 msgr2=0x55b63daca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63daca440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd618011ca0 tx=0x7fd618014000 comp rx=0 tx=0).stop 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 -- 172.21.15.110:0/831056554 shutdown_connections 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad9dc0 0x55b63daf14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad4c50 0x55b63dae5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 --2- 172.21.15.110:0/831056554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63daca440 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 -- 172.21.15.110:0/831056554 >> 172.21.15.110:0/831056554 conn(0x55b63dacf8d0 msgr2=0x55b63dac9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 -- 172.21.15.110:0/831056554 shutdown_connections 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.014+0000 7fd639777740 1 -- 172.21.15.110:0/831056554 wait complete. 2024-04-25T00:07:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.015+0000 7fd639777740 1 -- start start 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.015+0000 7fd639777740 1 Processor -- start 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.015+0000 7fd639777740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63dae3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd639777740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dae3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd639777740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dae4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd639777740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301a4b40 con 0x55b63dad4c50 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd639777740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301a5fb0 con 0x55b63dad9dc0 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63dae3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd639777740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301a5210 con 0x55b63dad4260 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63dae3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37780/0 (socket says 172.21.15.110:37780) 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 -- 172.21.15.110:0/3203742193 learned_addr learned my addr 172.21.15.110:0/3203742193 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd634f33640 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dae4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd639777740 10 client.0 _ll_register_callbacks cb 0x7fd6200bc1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dae3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 -- 172.21.15.110:0/3203742193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 msgr2=0x55b63dae4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dae4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 -- 172.21.15.110:0/3203742193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 msgr2=0x55b63dae3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dae3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 -- 172.21.15.110:0/3203742193 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b63dae4e70 con 0x55b63dad4260 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dae3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd62ffff640 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63dae3330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd61802d040 tx=0x7fd61802a470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c3d11b369d9374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.016+0000 7fd60f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd60f7fe640 1 -- 172.21.15.110:0/3203742193 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61802f040 con 0x55b63dad4260 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd60f7fe640 1 -- 172.21.15.110:0/3203742193 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd618027d10 con 0x55b63dad4260 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 1 -- 172.21.15.110:0/3203742193 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61802a670 con 0x55b63dad4260 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd60f7fe640 1 -- 172.21.15.110:0/3203742193 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd6180273c0 con 0x55b63dad4260 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 10 client.25270 fetch_fsmap learned FSMap version 32 2024-04-25T00:07:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 1 -- 172.21.15.110:0/3203742193 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b63db50d40 con 0x55b63dad4260 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd60f7fe640 1 -- 172.21.15.110:0/3203742193 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd61802a670 con 0x55b63dad4260 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd60f7fe640 1 -- 172.21.15.110:0/3203742193 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fd618032030 con 0x55b63dad4260 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 10 client.25270 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 10 client.25270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 20 client.25270 populate_metadata read hostname 'smithi110' 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b63db515b0 0x55b63db539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 4 client.25270 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.017+0000 7fd639777740 1 -- 172.21.15.110:0/3203742193 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b63dade590 con 0x55b63db515b0 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.018+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b63db515b0 0x55b63db539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.018+0000 7fd60f7fe640 10 client.25270 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.018+0000 7fd62f7fe640 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b63db515b0 0x55b63db539d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8321f9564a96bd8 server_cookie=9f34f1821ce60613 in_seq=0 out_seq=0 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd60f7fe640 1 -- 172.21.15.110:0/3203742193 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b63dade590 con 0x55b63db515b0 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd60f7fe640 10 client.25270 handle_command_reply: tid=0 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 client.25270 shutdown 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 -- 172.21.15.110:0/3203742193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 msgr2=0x55b63dae3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63dae3330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd61802d040 tx=0x7fd61802a470 comp rx=0 tx=0).stop 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 -- 172.21.15.110:0/3203742193 shutdown_connections 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b63db515b0 0x55b63db539d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b63dad9dc0 0x55b63dae4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd60f7fe640 0 client.25270 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b63dad4c50 0x55b63dae3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 --2- 172.21.15.110:0/3203742193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b63dad4260 0x55b63dae3330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 -- 172.21.15.110:0/3203742193 >> 172.21.15.110:0/3203742193 conn(0x55b63dacf8d0 msgr2=0x55b63daca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.035+0000 7fd639777740 1 -- 172.21.15.110:0/3203742193 shutdown_connections 2024-04-25T00:07:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.036+0000 7fd639777740 1 -- 172.21.15.110:0/3203742193 wait complete. 2024-04-25T00:07:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.036+0000 7fd639777740 20 client.25270 trim_cache size 0 max 16384 2024-04-25T00:07:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:07:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:07:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:07:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:07:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 -- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd61c0a2cb0 msgr2=0x7fd61c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd61c0a2cb0 0x7fd61c0a5170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd618009c90 tx=0x7fd618005f90 comp rx=0 tx=0).stop 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 -- 172.21.15.110:0/752947488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd630104c10 msgr2=0x7fd630197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd630104c10 0x7fd630197c50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd620009230 tx=0x7fd620003f80 comp rx=0 tx=0).stop 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 -- 172.21.15.110:0/752947488 shutdown_connections 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd61c0a2cb0 0x7fd61c0a5170 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd630107540 0x7fd63019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd630104c10 0x7fd630197c50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 --2- 172.21.15.110:0/752947488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6301022e0 0x7fd630197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 -- 172.21.15.110:0/752947488 >> 172.21.15.110:0/752947488 conn(0x7fd6300fbc10 msgr2=0x7fd630105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 -- 172.21.15.110:0/752947488 shutdown_connections 2024-04-25T00:07:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:37.037+0000 7fd636996640 1 -- 172.21.15.110:0/752947488 wait complete. 2024-04-25T00:07:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:37 smithi188 ceph-mon[33285]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 855 op/s 2024-04-25T00:07:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:37 smithi110 ceph-mon[29187]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 855 op/s 2024-04-25T00:07:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:37 smithi173 ceph-mon[32402]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 855 op/s 2024-04-25T00:07:40.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:39 smithi188 ceph-mon[33285]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 1.05k op/s 2024-04-25T00:07:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:39 smithi110 ceph-mon[29187]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 1.05k op/s 2024-04-25T00:07:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:39 smithi173 ceph-mon[32402]: pgmap v519: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 1.05k op/s 2024-04-25T00:07:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 275.0 s, 799.0 tps, lat 124.885 ms stddev 78.333 2024-04-25T00:07:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:41 smithi188 ceph-mon[33285]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 775 op/s 2024-04-25T00:07:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:41 smithi110 ceph-mon[29187]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 775 op/s 2024-04-25T00:07:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:41 smithi173 ceph-mon[32402]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 775 op/s 2024-04-25T00:07:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:43 smithi188 ceph-mon[33285]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 857 op/s 2024-04-25T00:07:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:43 smithi110 ceph-mon[29187]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 857 op/s 2024-04-25T00:07:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:43 smithi173 ceph-mon[32402]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 857 op/s 2024-04-25T00:07:46.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:45 smithi188 ceph-mon[33285]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 923 op/s 2024-04-25T00:07:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:45 smithi110 ceph-mon[29187]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 923 op/s 2024-04-25T00:07:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:45 smithi173 ceph-mon[32402]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 923 op/s 2024-04-25T00:07:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 280.0 s, 703.8 tps, lat 142.690 ms stddev 122.058 2024-04-25T00:07:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:47 smithi188 ceph-mon[33285]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 844 op/s 2024-04-25T00:07:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:47 smithi110 ceph-mon[29187]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 844 op/s 2024-04-25T00:07:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:47 smithi173 ceph-mon[32402]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 844 op/s 2024-04-25T00:07:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:49 smithi188 ceph-mon[33285]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-04-25T00:07:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:49 smithi110 ceph-mon[29187]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-04-25T00:07:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:49 smithi173 ceph-mon[32402]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 20 MiB/s wr, 1.04k op/s 2024-04-25T00:07:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.150+0000 7fe44a0b2640 1 -- 172.21.15.110:0/3099681722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4441006a0 msgr2=0x7fe444100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.150+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/3099681722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4441006a0 0x7fe444100ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe434009900 tx=0x7fe43402a410 comp rx=0 tx=0).stop 2024-04-25T00:07:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 -- 172.21.15.110:0/3099681722 shutdown_connections 2024-04-25T00:07:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/3099681722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe444102250 0x7fe444108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/3099681722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe444101890 0x7fe444101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/3099681722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4441006a0 0x7fe444100ac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 -- 172.21.15.110:0/3099681722 >> 172.21.15.110:0/3099681722 conn(0x7fe4440fbc10 msgr2=0x7fe4440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 -- 172.21.15.110:0/3099681722 shutdown_connections 2024-04-25T00:07:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 -- 172.21.15.110:0/3099681722 wait complete. 2024-04-25T00:07:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 Processor -- start 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.151+0000 7fe44a0b2640 1 -- start start 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe44a0b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4441006a0 0x7fe444195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe44a0b2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe444101890 0x7fe444195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe44a0b2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe444102250 0x7fe44419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe44a0b2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe44410b0d0 con 0x7fe4441006a0 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe44a0b2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe44410af50 con 0x7fe444102250 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe44a0b2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe44410b250 con 0x7fe444101890 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe444101890 0x7fe444195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe4437fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4441006a0 0x7fe444195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe444101890 0x7fe444195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:55528/0 (socket says 172.21.15.110:55528) 2024-04-25T00:07:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe442ffd640 1 -- 172.21.15.110:0/4170857507 learned_addr learned my addr 172.21.15.110:0/4170857507 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe443fff640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe444102250 0x7fe44419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe442ffd640 1 -- 172.21.15.110:0/4170857507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe444102250 msgr2=0x7fe44419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe442ffd640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe444102250 0x7fe44419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe442ffd640 1 -- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4441006a0 msgr2=0x7fe444195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe442ffd640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4441006a0 0x7fe444195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.152+0000 7fe442ffd640 1 -- 172.21.15.110:0/4170857507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4441a1770 con 0x7fe444101890 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.153+0000 7fe442ffd640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe444101890 0x7fe444195c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe438009860 tx=0x7fe438009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.153+0000 7fe440ff9640 1 -- 172.21.15.110:0/4170857507 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438004020 con 0x7fe444101890 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.153+0000 7fe44a0b2640 1 -- 172.21.15.110:0/4170857507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4441a1a60 con 0x7fe444101890 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.153+0000 7fe440ff9640 1 -- 172.21.15.110:0/4170857507 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe43800ed60 con 0x7fe444101890 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.153+0000 7fe440ff9640 1 -- 172.21.15.110:0/4170857507 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438003af0 con 0x7fe444101890 2024-04-25T00:07:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.153+0000 7fe44a0b2640 1 -- 172.21.15.110:0/4170857507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4441a2000 con 0x7fe444101890 2024-04-25T00:07:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe440ff9640 1 -- 172.21.15.110:0/4170857507 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe43801b020 con 0x7fe444101890 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe44cdbd740 1 Processor -- start 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe44cdbd740 1 -- start start 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a77e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a783340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a7839b0 0x55a22a785e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a22a6e1cb0 con 0x55a22a7839b0 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a22a68a5b0 con 0x55a22a77de50 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.156+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a22a6ddc70 con 0x55a22a77e840 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a77e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a77e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:48568/0 (socket says 172.21.15.110:48568) 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a7839b0 0x55a22a785e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 -- 172.21.15.110:0/86189311 learned_addr learned my addr 172.21.15.110:0/86189311 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe4437fe640 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a783340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe440ff9640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe42c0a2cb0 0x7fe42c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 -- 172.21.15.110:0/86189311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 msgr2=0x55a22a783340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a783340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 -- 172.21.15.110:0/86189311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a7839b0 msgr2=0x55a22a785e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a7839b0 0x55a22a785e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 -- 172.21.15.110:0/86189311 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a22a3f6210 con 0x55a22a77de50 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe442ffd640 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a7839b0 0x55a22a785e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe4437fe640 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a783340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe440ff9640 1 -- 172.21.15.110:0/4170857507 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe4380ce760 con 0x7fe444101890 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe42c0a2cb0 0x7fe42c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe443fff640 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a77e270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009fd0 tx=0x7fe43c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c3d91f54b7e4717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.157+0000 7fe429ffb640 1 -- 172.21.15.110:0/86189311 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c004050 con 0x55a22a77de50 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe429ffb640 1 -- 172.21.15.110:0/86189311 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe43c004720 con 0x55a22a77de50 2024-04-25T00:07:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe429ffb640 1 -- 172.21.15.110:0/86189311 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c019400 con 0x55a22a77de50 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 -- 172.21.15.110:0/86189311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 msgr2=0x55a22a77e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a77e270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009fd0 tx=0x7fe43c0118d0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe443fff640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe42c0a2cb0 0x7fe42c0a5170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe43c01d000 tx=0x7fe43c019760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 -- 172.21.15.110:0/86189311 shutdown_connections 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a7839b0 0x55a22a785e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a783340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/86189311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a77e270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 -- 172.21.15.110:0/86189311 >> 172.21.15.110:0/86189311 conn(0x55a22a7794c0 msgr2=0x55a22a7798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 -- 172.21.15.110:0/86189311 shutdown_connections 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.158+0000 7fe44cdbd740 1 -- 172.21.15.110:0/86189311 wait complete. 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 1 -- start start 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 1 Processor -- start 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77de50 0x55a22a7881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a7886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 0x55a22a78f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a22a786960 con 0x55a22a77e840 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a22a69db50 con 0x55a22a7839b0 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a22a786ae0 con 0x55a22a77de50 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe44cdbd740 10 client.0 _ll_register_callbacks cb 0x55a22a6d8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.159+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a7886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe443fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 0x55a22a78f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a7886f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40758/0 (socket says 172.21.15.110:40758) 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe442ffd640 1 -- 172.21.15.110:0/1185131166 learned_addr learned my addr 172.21.15.110:0/1185131166 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe4437fe640 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77de50 0x55a22a7881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe442ffd640 1 -- 172.21.15.110:0/1185131166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77de50 msgr2=0x55a22a7881b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe442ffd640 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77de50 0x55a22a7881b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe442ffd640 1 -- 172.21.15.110:0/1185131166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 msgr2=0x55a22a78f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe442ffd640 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 0x55a22a78f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe442ffd640 1 -- 172.21.15.110:0/1185131166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a22a4c2620 con 0x55a22a77e840 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe4437fe640 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77de50 0x55a22a7881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe442ffd640 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a7886f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe4380979c0 tx=0x7fe43800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a56638991f21cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe422ffd640 1 -- 172.21.15.110:0/1185131166 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43809a950 con 0x55a22a77e840 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe44cdbd740 1 -- 172.21.15.110:0/1185131166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43c004720 con 0x55a22a77e840 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe422ffd640 1 -- 172.21.15.110:0/1185131166 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe438097660 con 0x55a22a77e840 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe422ffd640 1 -- 172.21.15.110:0/1185131166 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe43800b680 con 0x55a22a77e840 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.160+0000 7fe44cdbd740 10 client.15723 fetch_fsmap learned FSMap version 32 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe44cdbd740 1 -- 172.21.15.110:0/1185131166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a22a792b60 con 0x55a22a77e840 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe422ffd640 1 -- 172.21.15.110:0/1185131166 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe43c004720 con 0x55a22a77e840 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe422ffd640 1 -- 172.21.15.110:0/1185131166 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe4380d0ce0 con 0x55a22a77e840 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe44cdbd740 10 client.15723 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe44cdbd740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:07:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe44cdbd740 20 client.15723 populate_metadata read hostname 'smithi110' 2024-04-25T00:07:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a22a7941e0 0x55a22a796610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe44cdbd740 4 client.15723 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:07:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe44cdbd740 1 -- 172.21.15.110:0/1185131166 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a22a6ef1f0 con 0x55a22a7941e0 2024-04-25T00:07:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.161+0000 7fe4437fe640 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a22a7941e0 0x55a22a796610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.162+0000 7fe422ffd640 10 client.15723 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.162+0000 7fe4437fe640 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a22a7941e0 0x55a22a796610 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d2ff26078e5de2c server_cookie=190166def20c34c3 in_seq=0 out_seq=0 2024-04-25T00:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.162+0000 7fe422ffd640 1 -- 172.21.15.110:0/1185131166 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a22a6ef1f0 con 0x55a22a7941e0 2024-04-25T00:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.162+0000 7fe422ffd640 10 client.15723 handle_command_reply: tid=0 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.162+0000 7fe44cdbd740 1 client.15723 shutdown 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 -- 172.21.15.110:0/1185131166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 msgr2=0x55a22a7886f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a7886f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe4380979c0 tx=0x7fe43800efd0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 -- 172.21.15.110:0/1185131166 shutdown_connections 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a22a7941e0 0x55a22a796610 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 0x55a22a78f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a7886f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe422ffd640 0 client.15723 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/1185131166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77de50 0x55a22a7881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 -- 172.21.15.110:0/1185131166 >> 172.21.15.110:0/1185131166 conn(0x55a22a7794c0 msgr2=0x55a22a7785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 -- 172.21.15.110:0/1185131166 shutdown_connections 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 1 -- 172.21.15.110:0/1185131166 wait complete. 2024-04-25T00:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.163+0000 7fe44cdbd740 20 client.15723 trim_cache size 0 max 16384 2024-04-25T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe44cdbd740 1 Processor -- start 2024-04-25T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe44cdbd740 1 -- start start 2024-04-25T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77de50 0x55a22a77c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a77ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 0x55a22a78f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441ad510 con 0x55a22a77de50 2024-04-25T00:07:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441aade0 con 0x55a22a7839b0 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441a8c30 con 0x55a22a77e840 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe4437fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77de50 0x55a22a77c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe4437fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77de50 0x55a22a77c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40774/0 (socket says 172.21.15.110:40774) 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe4437fe640 1 -- 172.21.15.110:0/466319352 learned_addr learned my addr 172.21.15.110:0/466319352 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe442ffd640 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a77ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe443fff640 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 0x55a22a78f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe4437fe640 1 -- 172.21.15.110:0/466319352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 msgr2=0x55a22a77ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe4437fe640 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a77ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe4437fe640 1 -- 172.21.15.110:0/466319352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 msgr2=0x55a22a78f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe4437fe640 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 0x55a22a78f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.185+0000 7fe4437fe640 1 -- 172.21.15.110:0/466319352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a22a77ec20 con 0x55a22a77de50 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.186+0000 7fe4437fe640 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77de50 0x55a22a77c950 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe4441016f0 tx=0x7fe434009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a16e0b9dfc65d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.186+0000 7fe4297fa640 1 -- 172.21.15.110:0/466319352 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43404a020 con 0x55a22a77de50 2024-04-25T00:07:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.186+0000 7fe4297fa640 1 -- 172.21.15.110:0/466319352 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4340463b0 con 0x55a22a77de50 2024-04-25T00:07:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.186+0000 7fe4297fa640 1 -- 172.21.15.110:0/466319352 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434035040 con 0x55a22a77de50 2024-04-25T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 -- 172.21.15.110:0/466319352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77de50 msgr2=0x55a22a77c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77de50 0x55a22a77c950 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe4441016f0 tx=0x7fe434009260 comp rx=0 tx=0).stop 2024-04-25T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 -- 172.21.15.110:0/466319352 shutdown_connections 2024-04-25T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a7839b0 0x55a22a78f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a77e840 0x55a22a77ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/466319352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77de50 0x55a22a77c950 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 -- 172.21.15.110:0/466319352 >> 172.21.15.110:0/466319352 conn(0x55a22a7794c0 msgr2=0x55a22a77c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 -- 172.21.15.110:0/466319352 shutdown_connections 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.187+0000 7fe44cdbd740 1 -- 172.21.15.110:0/466319352 wait complete. 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 1 -- start start 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 1 Processor -- start 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a78d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a78d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a7839b0 0x55a22a78e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441a92a0 con 0x55a22a77e840 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441aa710 con 0x55a22a77de50 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441a9970 con 0x55a22a7839b0 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.188+0000 7fe44cdbd740 10 client.0 _ll_register_callbacks cb 0x7fe4380d4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:07:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a78d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe443fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a7839b0 0x55a22a78e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a78d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40786/0 (socket says 172.21.15.110:40786) 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 -- 172.21.15.110:0/839386779 learned_addr learned my addr 172.21.15.110:0/839386779 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe4437fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a78d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 -- 172.21.15.110:0/839386779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a7839b0 msgr2=0x55a22a78e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a7839b0 0x55a22a78e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 -- 172.21.15.110:0/839386779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 msgr2=0x55a22a78d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a78d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 -- 172.21.15.110:0/839386779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a22a78ee50 con 0x55a22a77e840 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe443fff640 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a7839b0 0x55a22a78e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe442ffd640 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a78d8a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe43800e490 tx=0x7fe438003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329142dd733563b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:07:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe422ffd640 1 -- 172.21.15.110:0/839386779 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380b3070 con 0x55a22a77e840 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe422ffd640 1 -- 172.21.15.110:0/839386779 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe43809a710 con 0x55a22a77e840 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.189+0000 7fe44cdbd740 1 -- 172.21.15.110:0/839386779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4340463b0 con 0x55a22a77e840 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe422ffd640 1 -- 172.21.15.110:0/839386779 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe438079e60 con 0x55a22a77e840 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe44cdbd740 10 client.15729 fetch_fsmap learned FSMap version 32 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe44cdbd740 1 -- 172.21.15.110:0/839386779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a22a7facc0 con 0x55a22a77e840 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe422ffd640 1 -- 172.21.15.110:0/839386779 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4340463b0 con 0x55a22a77e840 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe422ffd640 1 -- 172.21.15.110:0/839386779 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe438096d40 con 0x55a22a77e840 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe44cdbd740 10 client.15729 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe44cdbd740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe44cdbd740 20 client.15729 populate_metadata read hostname 'smithi110' 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a22a7941e0 0x55a22a7fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe44cdbd740 4 client.15729 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe44cdbd740 1 -- 172.21.15.110:0/839386779 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a22a6e9540 con 0x55a22a7941e0 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.190+0000 7fe4437fe640 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a22a7941e0 0x55a22a7fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.191+0000 7fe4437fe640 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a22a7941e0 0x55a22a7fd510 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604eb6521e0e54bc server_cookie=4ff7cc5deb3685cc in_seq=0 out_seq=0 2024-04-25T00:07:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.191+0000 7fe422ffd640 10 client.15729 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe422ffd640 1 -- 172.21.15.110:0/839386779 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a22a6e9540 con 0x55a22a7941e0 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe422ffd640 10 client.15729 handle_command_reply: tid=0 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 client.15729 shutdown 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 -- 172.21.15.110:0/839386779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 msgr2=0x55a22a78d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a78d8a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe43800e490 tx=0x7fe438003ef0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 -- 172.21.15.110:0/839386779 shutdown_connections 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a22a7941e0 0x55a22a7fd510 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a22a7839b0 0x55a22a78e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a22a77e840 0x55a22a78d8a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 --2- 172.21.15.110:0/839386779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a22a77de50 0x55a22a78d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe422ffd640 0 client.15729 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:07:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 -- 172.21.15.110:0/839386779 >> 172.21.15.110:0/839386779 conn(0x55a22a7794c0 msgr2=0x55a22a77cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 -- 172.21.15.110:0/839386779 shutdown_connections 2024-04-25T00:07:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 1 -- 172.21.15.110:0/839386779 wait complete. 2024-04-25T00:07:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.206+0000 7fe44cdbd740 20 client.15729 trim_cache size 0 max 16384 2024-04-25T00:07:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:07:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:07:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:07:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:07:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 -- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe42c0a2cb0 msgr2=0x7fe42c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe42c0a2cb0 0x7fe42c0a5170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe43c01d000 tx=0x7fe43c019760 comp rx=0 tx=0).stop 2024-04-25T00:07:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 -- 172.21.15.110:0/4170857507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe444101890 msgr2=0x7fe444195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe444101890 0x7fe444195c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe438009860 tx=0x7fe438009d20 comp rx=0 tx=0).stop 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 -- 172.21.15.110:0/4170857507 shutdown_connections 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe42c0a2cb0 0x7fe42c0a5170 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe444102250 0x7fe44419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe444101890 0x7fe444195c80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 --2- 172.21.15.110:0/4170857507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4441006a0 0x7fe444195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 -- 172.21.15.110:0/4170857507 >> 172.21.15.110:0/4170857507 conn(0x7fe4440fbc10 msgr2=0x7fe444104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 -- 172.21.15.110:0/4170857507 shutdown_connections 2024-04-25T00:07:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:07:50.207+0000 7fe44a0b2640 1 -- 172.21.15.110:0/4170857507 wait complete. 2024-04-25T00:07:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 285.0 s, 783.6 tps, lat 127.244 ms stddev 79.147 2024-04-25T00:07:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:51 smithi188 ceph-mon[33285]: pgmap v525: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-25T00:07:52.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:51 smithi110 ceph-mon[29187]: pgmap v525: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-25T00:07:52.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:51 smithi173 ceph-mon[32402]: pgmap v525: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-25T00:07:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:53 smithi188 ceph-mon[33285]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 18 MiB/s wr, 994 op/s 2024-04-25T00:07:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:53 smithi110 ceph-mon[29187]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 18 MiB/s wr, 994 op/s 2024-04-25T00:07:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:53 smithi173 ceph-mon[32402]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 18 MiB/s wr, 994 op/s 2024-04-25T00:07:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:55 smithi188 ceph-mon[33285]: pgmap v527: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 982 op/s 2024-04-25T00:07:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:55 smithi110 ceph-mon[29187]: pgmap v527: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 982 op/s 2024-04-25T00:07:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:55 smithi173 ceph-mon[32402]: pgmap v527: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 982 op/s 2024-04-25T00:07:56.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 290.0 s, 796.6 tps, lat 125.378 ms stddev 77.805 2024-04-25T00:07:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:57 smithi188 ceph-mon[33285]: pgmap v528: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 793 op/s 2024-04-25T00:07:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:57 smithi110 ceph-mon[29187]: pgmap v528: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 793 op/s 2024-04-25T00:07:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:57 smithi173 ceph-mon[32402]: pgmap v528: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 793 op/s 2024-04-25T00:08:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:07:59 smithi188 ceph-mon[33285]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2024-04-25T00:08:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:07:59 smithi110 ceph-mon[29187]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2024-04-25T00:08:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:07:59 smithi173 ceph-mon[32402]: pgmap v529: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2024-04-25T00:08:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 295.0 s, 692.0 tps, lat 145.486 ms stddev 117.519 2024-04-25T00:08:02.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:01 smithi188 ceph-mon[33285]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 953 op/s 2024-04-25T00:08:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:01 smithi110 ceph-mon[29187]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 953 op/s 2024-04-25T00:08:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:01 smithi173 ceph-mon[32402]: pgmap v530: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 953 op/s 2024-04-25T00:08:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.338+0000 7f38cbf77640 1 -- 172.21.15.110:0/1854038199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c4102250 msgr2=0x7f38c4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.338+0000 7f38cbf77640 1 --2- 172.21.15.110:0/1854038199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c4102250 0x7f38c4108ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009ca0 tx=0x7f38bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.338+0000 7f38cbf77640 1 -- 172.21.15.110:0/1854038199 shutdown_connections 2024-04-25T00:08:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.338+0000 7f38cbf77640 1 --2- 172.21.15.110:0/1854038199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c4102250 0x7f38c4108ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.338+0000 7f38cbf77640 1 --2- 172.21.15.110:0/1854038199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c4101890 0x7f38c4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.338+0000 7f38cbf77640 1 --2- 172.21.15.110:0/1854038199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c41006a0 0x7f38c4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.338+0000 7f38cbf77640 1 -- 172.21.15.110:0/1854038199 >> 172.21.15.110:0/1854038199 conn(0x7f38c40fbc10 msgr2=0x7f38c40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.338+0000 7f38cbf77640 1 -- 172.21.15.110:0/1854038199 shutdown_connections 2024-04-25T00:08:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.339+0000 7f38cbf77640 1 -- 172.21.15.110:0/1854038199 wait complete. 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.339+0000 7f38cbf77640 1 Processor -- start 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.339+0000 7f38cbf77640 1 -- start start 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38cbf77640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c41006a0 0x7f38c4196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38cbf77640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c4101890 0x7f38c4196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38cbf77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c4102250 0x7f38c419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38cbf77640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410c220 con 0x7f38c4101890 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38cbf77640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410c0a0 con 0x7f38c41006a0 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38cbf77640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410c3a0 con 0x7f38c4102250 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9512640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c4101890 0x7f38c4196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c41006a0 0x7f38c4196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c41006a0 0x7f38c4196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:46772/0 (socket says 172.21.15.110:46772) 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 -- 172.21.15.110:0/3995022740 learned_addr learned my addr 172.21.15.110:0/3995022740 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38ca514640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c4102250 0x7f38c419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 -- 172.21.15.110:0/3995022740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c4102250 msgr2=0x7f38c419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c4102250 0x7f38c419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 -- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c4101890 msgr2=0x7f38c4196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c4101890 0x7f38c4196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 -- 172.21.15.110:0/3995022740 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c4068ef0 con 0x7f38c41006a0 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9512640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c4101890 0x7f38c4196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.340+0000 7f38c9d13640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c41006a0 0x7f38c4196610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38c000e9c0 tx=0x7f38c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.341+0000 7f38baffd640 1 -- 172.21.15.110:0/3995022740 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c000cd80 con 0x7f38c41006a0 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.341+0000 7f38baffd640 1 -- 172.21.15.110:0/3995022740 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c00035f0 con 0x7f38c41006a0 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.341+0000 7f38baffd640 1 -- 172.21.15.110:0/3995022740 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0010690 con 0x7f38c41006a0 2024-04-25T00:08:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.341+0000 7f38cbf77640 1 -- 172.21.15.110:0/3995022740 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c40691e0 con 0x7f38c41006a0 2024-04-25T00:08:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.341+0000 7f38cbf77640 1 -- 172.21.15.110:0/3995022740 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c40696e0 con 0x7f38c41006a0 2024-04-25T00:08:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.344+0000 7f38ced66740 1 Processor -- start 2024-04-25T00:08:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.344+0000 7f38ced66740 1 -- start start 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ced66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bbe50 0x56413d0bc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ced66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ced66740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ced66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56413d01fcb0 con 0x56413d0bbe50 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ced66740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56413cfc85b0 con 0x56413d0c19b0 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ced66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56413d01bc70 con 0x56413d0bc840 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ca514640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bbe50 0x56413d0bc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ca514640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bbe50 0x56413d0bc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43534/0 (socket says 172.21.15.110:43534) 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ca514640 1 -- 172.21.15.110:0/1863189271 learned_addr learned my addr 172.21.15.110:0/1863189271 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38c9512640 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38baffd640 1 -- 172.21.15.110:0/3995022740 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f38c00037a0 con 0x7f38c41006a0 2024-04-25T00:08:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38c9512640 1 -- 172.21.15.110:0/1863189271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 msgr2=0x56413d0c1340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:08:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38c9512640 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0c1340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38c9512640 1 -- 172.21.15.110:0/1863189271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bbe50 msgr2=0x56413d0bc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38c9512640 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bbe50 0x56413d0bc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38c9512640 1 -- 172.21.15.110:0/1863189271 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56413cd34210 con 0x56413d0c19b0 2024-04-25T00:08:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.345+0000 7f38ca514640 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bbe50 0x56413d0bc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:08:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.346+0000 7f38c9512640 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0c3e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009c00 tx=0x7f38b40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a28c9475ec9dbe63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.346+0000 7f38baffd640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f38b00a2c30 0x7f38b00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.346+0000 7f38affff640 1 -- 172.21.15.110:0/1863189271 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4012450 con 0x56413d0c19b0 2024-04-25T00:08:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.346+0000 7f38affff640 1 -- 172.21.15.110:0/1863189271 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38b4012aa0 con 0x56413d0c19b0 2024-04-25T00:08:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.346+0000 7f38affff640 1 -- 172.21.15.110:0/1863189271 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400fa30 con 0x56413d0c19b0 2024-04-25T00:08:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.346+0000 7f38ca514640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f38b00a2c30 0x7f38b00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.347+0000 7f38baffd640 1 -- 172.21.15.110:0/3995022740 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f38c00cf730 con 0x7f38c41006a0 2024-04-25T00:08:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ca514640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f38b00a2c30 0x7f38b00a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005e30 tx=0x7f38bc031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 -- 172.21.15.110:0/1863189271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 msgr2=0x56413d0c3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0c3e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009c00 tx=0x7f38b40117b0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 -- 172.21.15.110:0/1863189271 shutdown_connections 2024-04-25T00:08:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0c3e40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 --2- 172.21.15.110:0/1863189271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bbe50 0x56413d0bc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 -- 172.21.15.110:0/1863189271 >> 172.21.15.110:0/1863189271 conn(0x56413d0b74c0 msgr2=0x56413d0b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 -- 172.21.15.110:0/1863189271 shutdown_connections 2024-04-25T00:08:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 -- 172.21.15.110:0/1863189271 wait complete. 2024-04-25T00:08:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.348+0000 7f38ced66740 1 -- start start 2024-04-25T00:08:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ced66740 1 Processor -- start 2024-04-25T00:08:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ced66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ced66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bc840 0x56413d0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ced66740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ced66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56413d0c4820 con 0x56413d0bc840 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ced66740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56413cfdbb50 con 0x56413d0c19b0 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ced66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56413d0c49a0 con 0x56413d0bbe50 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ced66740 10 client.0 _ll_register_callbacks cb 0x56413d016ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ca514640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ca514640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:46798/0 (socket says 172.21.15.110:46798) 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.349+0000 7f38ca514640 1 -- 172.21.15.110:0/2389345142 learned_addr learned my addr 172.21.15.110:0/2389345142 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ca514640 1 -- 172.21.15.110:0/2389345142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 msgr2=0x56413d0c60a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ca514640 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0c60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ca514640 1 -- 172.21.15.110:0/2389345142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bc840 msgr2=0x56413d0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ca514640 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bc840 0x56413d0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ca514640 1 -- 172.21.15.110:0/2389345142 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56413ce00620 con 0x56413d0c19b0 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ca514640 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0cb5a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38bc031910 tx=0x7f38bc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2556c50a343d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ad7fa640 1 -- 172.21.15.110:0/2389345142 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0098c0 con 0x56413d0c19b0 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.350+0000 7f38ad7fa640 1 -- 172.21.15.110:0/2389345142 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f38bc004250 con 0x56413d0c19b0 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ced66740 1 -- 172.21.15.110:0/2389345142 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b4012aa0 con 0x56413d0c19b0 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ad7fa640 1 -- 172.21.15.110:0/2389345142 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f38bc004450 con 0x56413d0c19b0 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ced66740 10 client.25325 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ced66740 1 -- 172.21.15.110:0/2389345142 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56413d0d0a60 con 0x56413d0c19b0 2024-04-25T00:08:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ad7fa640 1 -- 172.21.15.110:0/2389345142 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38bc036040 con 0x56413d0c19b0 2024-04-25T00:08:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ad7fa640 1 -- 172.21.15.110:0/2389345142 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f38bc062070 con 0x56413d0c19b0 2024-04-25T00:08:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ced66740 10 client.25325 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ced66740 10 client.25325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ced66740 20 client.25325 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ced66740 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56413d0d2190 0x56413d0d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.351+0000 7f38ced66740 4 client.25325 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.352+0000 7f38ced66740 1 -- 172.21.15.110:0/2389345142 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f38b4012aa0 con 0x56413d0d2190 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.352+0000 7f38c9d13640 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56413d0d2190 0x56413d0d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.352+0000 7f38c9d13640 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56413d0d2190 0x56413d0d4650 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c2a40fce3a1339 server_cookie=5b6956e853ed7d12 in_seq=0 out_seq=0 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.352+0000 7f38ad7fa640 10 client.25325 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.353+0000 7f38ad7fa640 1 -- 172.21.15.110:0/2389345142 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f38b4012aa0 con 0x56413d0d2190 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ad7fa640 10 client.25325 handle_command_reply: tid=0 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 client.25325 shutdown 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 -- 172.21.15.110:0/2389345142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 msgr2=0x56413d0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0cb5a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38bc031910 tx=0x7f38bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 -- 172.21.15.110:0/2389345142 shutdown_connections 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56413d0d2190 0x56413d0d4650 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0c19b0 0x56413d0cb5a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0bc840 0x56413d0c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 --2- 172.21.15.110:0/2389345142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 -- 172.21.15.110:0/2389345142 >> 172.21.15.110:0/2389345142 conn(0x56413d0b74c0 msgr2=0x56413d0b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ad7fa640 0 client.25325 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 -- 172.21.15.110:0/2389345142 shutdown_connections 2024-04-25T00:08:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.354+0000 7f38ced66740 1 -- 172.21.15.110:0/2389345142 wait complete. 2024-04-25T00:08:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.355+0000 7f38ced66740 20 client.25325 trim_cache size 0 max 16384 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38ced66740 1 Processor -- start 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38ced66740 1 -- start start 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38ced66740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bbe50 0x56413d0bdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38ced66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38ced66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 0x56413d0ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38ced66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41a8f80 con 0x56413d0c19b0 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38ced66740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41a6850 con 0x56413d0bbe50 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38ced66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41a46a0 con 0x56413d0bc840 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38c9512640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38c9512640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0be0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:43658/0 (socket says 172.21.15.110:43658) 2024-04-25T00:08:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.379+0000 7f38c9512640 1 -- 172.21.15.110:0/740531337 learned_addr learned my addr 172.21.15.110:0/740531337 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.380+0000 7f38c9d13640 1 --2- 172.21.15.110:0/740531337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bbe50 0x56413d0bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.380+0000 7f38c9512640 1 -- 172.21.15.110:0/740531337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bbe50 msgr2=0x56413d0bdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.380+0000 7f38c9512640 1 --2- 172.21.15.110:0/740531337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bbe50 0x56413d0bdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.380+0000 7f38c9512640 1 -- 172.21.15.110:0/740531337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 msgr2=0x56413d0ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.380+0000 7f38c9512640 1 --2- 172.21.15.110:0/740531337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 0x56413d0ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.380+0000 7f38c9512640 1 -- 172.21.15.110:0/740531337 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56413d0b8230 con 0x56413d0bc840 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.380+0000 7f38c9512640 1 --2- 172.21.15.110:0/740531337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0be0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f38b401c040 tx=0x7f38b40095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cb3ead05d2bf775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.381+0000 7f38af7fe640 1 -- 172.21.15.110:0/740531337 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4005030 con 0x56413d0bc840 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.381+0000 7f38af7fe640 1 -- 172.21.15.110:0/740531337 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38b4004660 con 0x56413d0bc840 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.381+0000 7f38af7fe640 1 -- 172.21.15.110:0/740531337 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b40126b0 con 0x56413d0bc840 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.382+0000 7f38ced66740 1 -- 172.21.15.110:0/740531337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 msgr2=0x56413d0be0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.382+0000 7f38ced66740 1 --2- 172.21.15.110:0/740531337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0be0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f38b401c040 tx=0x7f38b40095f0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.383+0000 7f38ced66740 1 -- 172.21.15.110:0/740531337 shutdown_connections 2024-04-25T00:08:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.383+0000 7f38ced66740 1 --2- 172.21.15.110:0/740531337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 0x56413d0ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.383+0000 7f38ced66740 1 --2- 172.21.15.110:0/740531337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bc840 0x56413d0be0e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.383+0000 7f38ced66740 1 --2- 172.21.15.110:0/740531337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bbe50 0x56413d0bdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.383+0000 7f38ced66740 1 -- 172.21.15.110:0/740531337 >> 172.21.15.110:0/740531337 conn(0x56413d0b74c0 msgr2=0x56413d0b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.383+0000 7f38ced66740 1 -- 172.21.15.110:0/740531337 shutdown_connections 2024-04-25T00:08:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.384+0000 7f38ced66740 1 -- 172.21.15.110:0/740531337 wait complete. 2024-04-25T00:08:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.384+0000 7f38ced66740 1 -- start start 2024-04-25T00:08:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.384+0000 7f38ced66740 1 Processor -- start 2024-04-25T00:08:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38ced66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0e3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38ced66740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bc840 0x56413d0e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38ced66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 0x56413d0e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38c9d13640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0e3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38c9d13640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0e3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:43666/0 (socket says 172.21.15.110:43666) 2024-04-25T00:08:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38c9d13640 1 -- 172.21.15.110:0/3675154384 learned_addr learned my addr 172.21.15.110:0/3675154384 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41a4d10 con 0x56413d0c19b0 2024-04-25T00:08:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41a6180 con 0x56413d0bc840 2024-04-25T00:08:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41a53e0 con 0x56413d0bbe50 2024-04-25T00:08:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38ca514640 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 0x56413d0e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.385+0000 7f38ced66740 10 client.0 _ll_register_callbacks cb 0x7f38c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38c9d13640 1 -- 172.21.15.110:0/3675154384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bc840 msgr2=0x56413d0e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38c9d13640 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bc840 0x56413d0e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38c9d13640 1 -- 172.21.15.110:0/3675154384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 msgr2=0x56413d0e4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38c9d13640 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 0x56413d0e4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38c9d13640 1 -- 172.21.15.110:0/3675154384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56413d0e54c0 con 0x56413d0bbe50 2024-04-25T00:08:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38c9d13640 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0e3980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38c000cfd0 tx=0x7f38c0003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dda9eec00861b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:08:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38ad7fa640 1 -- 172.21.15.110:0/3675154384 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c009c660 con 0x56413d0bbe50 2024-04-25T00:08:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38ad7fa640 1 -- 172.21.15.110:0/3675154384 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f38c00985d0 con 0x56413d0bbe50 2024-04-25T00:08:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.386+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b4004660 con 0x56413d0bbe50 2024-04-25T00:08:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.387+0000 7f38ad7fa640 1 -- 172.21.15.110:0/3675154384 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f38c0066050 con 0x56413d0bbe50 2024-04-25T00:08:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.387+0000 7f38ced66740 10 client.25300 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.387+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56413d138b30 con 0x56413d0bbe50 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.388+0000 7f38ad7fa640 1 -- 172.21.15.110:0/3675154384 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38b4004660 con 0x56413d0bbe50 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.388+0000 7f38ad7fa640 1 -- 172.21.15.110:0/3675154384 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f38c001fbf0 con 0x56413d0bbe50 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.388+0000 7f38ced66740 10 client.25300 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.388+0000 7f38ced66740 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.388+0000 7f38ced66740 20 client.25300 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.388+0000 7f38ced66740 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56413d1394e0 0x56413d13b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.388+0000 7f38ced66740 4 client.25300 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:08:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.388+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56413d02d1f0 con 0x56413d1394e0 2024-04-25T00:08:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.389+0000 7f38c9512640 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56413d1394e0 0x56413d13b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.390+0000 7f38ad7fa640 10 client.25300 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.390+0000 7f38c9512640 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56413d1394e0 0x56413d13b900 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ccf0f4ea8b5080 server_cookie=1963e800f699eefc in_seq=0 out_seq=0 2024-04-25T00:08:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ad7fa640 1 -- 172.21.15.110:0/3675154384 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56413d02d1f0 con 0x56413d1394e0 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ad7fa640 10 client.25300 handle_command_reply: tid=0 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 client.25300 shutdown 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 msgr2=0x56413d0e3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0e3980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38c000cfd0 tx=0x7f38c0003e70 comp rx=0 tx=0).stop 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 shutdown_connections 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56413d1394e0 0x56413d13b900 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ad7fa640 0 client.25300 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56413d0c19b0 0x56413d0e4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56413d0bc840 0x56413d0e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 --2- 172.21.15.110:0/3675154384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56413d0bbe50 0x56413d0e3980 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 >> 172.21.15.110:0/3675154384 conn(0x56413d0b74c0 msgr2=0x56413d0c5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 shutdown_connections 2024-04-25T00:08:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 1 -- 172.21.15.110:0/3675154384 wait complete. 2024-04-25T00:08:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.406+0000 7f38ced66740 20 client.25300 trim_cache size 0 max 16384 2024-04-25T00:08:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:08:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:08:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:08:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 -- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f38b00a2c30 msgr2=0x7f38b00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f38b00a2c30 0x7f38b00a50f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005e30 tx=0x7f38bc031000 comp rx=0 tx=0).stop 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 -- 172.21.15.110:0/3995022740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c41006a0 msgr2=0x7f38c4196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c41006a0 0x7f38c4196610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38c000e9c0 tx=0x7f38c000ee80 comp rx=0 tx=0).stop 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 -- 172.21.15.110:0/3995022740 shutdown_connections 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f38b00a2c30 0x7f38b00a50f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38c4102250 0x7f38c419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c4101890 0x7f38c4196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 --2- 172.21.15.110:0/3995022740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38c41006a0 0x7f38c4196610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 -- 172.21.15.110:0/3995022740 >> 172.21.15.110:0/3995022740 conn(0x7f38c40fbc10 msgr2=0x7f38c4104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 -- 172.21.15.110:0/3995022740 shutdown_connections 2024-04-25T00:08:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:03.407+0000 7f38cbf77640 1 -- 172.21.15.110:0/3995022740 wait complete. 2024-04-25T00:08:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:03 smithi188 ceph-mon[33285]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-04-25T00:08:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:03 smithi110 ceph-mon[29187]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-04-25T00:08:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:03 smithi173 ceph-mon[32402]: pgmap v531: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-04-25T00:08:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:05 smithi188 ceph-mon[33285]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 970 op/s 2024-04-25T00:08:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:05 smithi110 ceph-mon[29187]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 970 op/s 2024-04-25T00:08:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:05 smithi173 ceph-mon[32402]: pgmap v532: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 970 op/s 2024-04-25T00:08:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 300.0 s, 797.2 tps, lat 124.789 ms stddev 80.503 2024-04-25T00:08:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:07 smithi188 ceph-mon[33285]: pgmap v533: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 879 op/s 2024-04-25T00:08:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:07 smithi110 ceph-mon[29187]: pgmap v533: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 879 op/s 2024-04-25T00:08:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:07 smithi173 ceph-mon[32402]: pgmap v533: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 879 op/s 2024-04-25T00:08:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:09 smithi188 ceph-mon[33285]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.16k op/s 2024-04-25T00:08:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:09 smithi110 ceph-mon[29187]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.16k op/s 2024-04-25T00:08:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:09 smithi173 ceph-mon[32402]: pgmap v534: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.16k op/s 2024-04-25T00:08:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 305.0 s, 775.6 tps, lat 128.949 ms stddev 80.422 2024-04-25T00:08:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:11 smithi188 ceph-mon[33285]: pgmap v535: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 925 op/s 2024-04-25T00:08:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:11 smithi110 ceph-mon[29187]: pgmap v535: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 925 op/s 2024-04-25T00:08:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:11 smithi173 ceph-mon[32402]: pgmap v535: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 925 op/s 2024-04-25T00:08:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:13 smithi188 ceph-mon[33285]: pgmap v536: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 960 op/s 2024-04-25T00:08:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:13 smithi110 ceph-mon[29187]: pgmap v536: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 960 op/s 2024-04-25T00:08:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:13 smithi173 ceph-mon[32402]: pgmap v536: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 960 op/s 2024-04-25T00:08:15.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:14 smithi173 ceph-mon[32402]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 984 op/s 2024-04-25T00:08:15.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:14 smithi188 ceph-mon[33285]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 984 op/s 2024-04-25T00:08:15.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:14 smithi110 ceph-mon[29187]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 984 op/s 2024-04-25T00:08:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 -- 172.21.15.110:0/2516280690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c101890 msgr2=0x7f5e6c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 --2- 172.21.15.110:0/2516280690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c101890 0x7f5e6c101d10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009860 tx=0x7f5e6002a340 comp rx=0 tx=0).stop 2024-04-25T00:08:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 -- 172.21.15.110:0/2516280690 shutdown_connections 2024-04-25T00:08:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 --2- 172.21.15.110:0/2516280690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e6c102250 0x7f5e6c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 --2- 172.21.15.110:0/2516280690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c101890 0x7f5e6c101d10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 --2- 172.21.15.110:0/2516280690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6c1006a0 0x7f5e6c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 -- 172.21.15.110:0/2516280690 >> 172.21.15.110:0/2516280690 conn(0x7f5e6c0fbc10 msgr2=0x7f5e6c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 -- 172.21.15.110:0/2516280690 shutdown_connections 2024-04-25T00:08:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.532+0000 7f5e71390640 1 -- 172.21.15.110:0/2516280690 wait complete. 2024-04-25T00:08:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.533+0000 7f5e71390640 1 Processor -- start 2024-04-25T00:08:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.533+0000 7f5e71390640 1 -- start start 2024-04-25T00:08:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.533+0000 7f5e71390640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c1006a0 0x7f5e6c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e71390640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e6c101890 0x7f5e6c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e71390640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6c102250 0x7f5e6c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e71390640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c069950 con 0x7f5e6c1006a0 2024-04-25T00:08:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e71390640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c0697d0 con 0x7f5e6c101890 2024-04-25T00:08:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e71390640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c069ad0 con 0x7f5e6c102250 2024-04-25T00:08:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c1006a0 0x7f5e6c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c1006a0 0x7f5e6c197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58838/0 (socket says 172.21.15.110:58838) 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6affd640 1 -- 172.21.15.110:0/4089769598 learned_addr learned my addr 172.21.15.110:0/4089769598 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6b7fe640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6c102250 0x7f5e6c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e6c101890 0x7f5e6c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6b7fe640 1 -- 172.21.15.110:0/4089769598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e6c101890 msgr2=0x7f5e6c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6b7fe640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e6c101890 0x7f5e6c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6b7fe640 1 -- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c1006a0 msgr2=0x7f5e6c197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6b7fe640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c1006a0 0x7f5e6c197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6b7fe640 1 -- 172.21.15.110:0/4089769598 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6c19f5a0 con 0x7f5e6c102250 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6affd640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c1006a0 0x7f5e6c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.534+0000 7f5e6b7fe640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6c102250 0x7f5e6c19ef30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400b670 tx=0x7f5e6400bb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.535+0000 7f5e53fff640 1 -- 172.21.15.110:0/4089769598 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64003970 con 0x7f5e6c102250 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.535+0000 7f5e53fff640 1 -- 172.21.15.110:0/4089769598 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e6400a850 con 0x7f5e6c102250 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.535+0000 7f5e53fff640 1 -- 172.21.15.110:0/4089769598 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e640044e0 con 0x7f5e6c102250 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.535+0000 7f5e71390640 1 -- 172.21.15.110:0/4089769598 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6c10b6b0 con 0x7f5e6c102250 2024-04-25T00:08:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.535+0000 7f5e71390640 1 -- 172.21.15.110:0/4089769598 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6c10bb80 con 0x7f5e6c102250 2024-04-25T00:08:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.538+0000 7f5e53fff640 1 -- 172.21.15.110:0/4089769598 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5e640046a0 con 0x7f5e6c102250 2024-04-25T00:08:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.538+0000 7f5e74178740 1 Processor -- start 2024-04-25T00:08:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.538+0000 7f5e74178740 1 -- start start 2024-04-25T00:08:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e74178740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af784d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e74178740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784d840 0x560af7852340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e74178740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af78529b0 0x560af7854e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e74178740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560af77b0cb0 con 0x560af78529b0 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e74178740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560af77595b0 con 0x560af784d840 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e74178740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560af77acc70 con 0x560af784ce50 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e53fff640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5e540a2c30 0x7f5e540a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784d840 0x560af7852340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784d840 0x560af7852340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:38386/0 (socket says 172.21.15.110:38386) 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6affd640 1 -- 172.21.15.110:0/4042580915 learned_addr learned my addr 172.21.15.110:0/4042580915 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e53fff640 1 -- 172.21.15.110:0/4089769598 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5e640d31a0 con 0x7f5e6c102250 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af78529b0 0x560af7854e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af784d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6b7fe640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5e540a2c30 0x7f5e540a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6affd640 1 -- 172.21.15.110:0/4042580915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 msgr2=0x560af784d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6affd640 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af784d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6affd640 1 -- 172.21.15.110:0/4042580915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af78529b0 msgr2=0x560af7854e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6affd640 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af78529b0 0x560af7854e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.539+0000 7f5e6affd640 1 -- 172.21.15.110:0/4042580915 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af74c5210 con 0x560af784d840 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.540+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af78529b0 0x560af7854e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.540+0000 7f5e6affd640 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784d840 0x560af7852340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0115c0 tx=0x7f5e5c011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35ed48b102ca50c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.540+0000 7f5e517fa640 1 -- 172.21.15.110:0/4042580915 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c005020 con 0x560af784d840 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.540+0000 7f5e6b7fe640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5e540a2c30 0x7f5e540a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5e640a8170 tx=0x7f5e64025000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.540+0000 7f5e517fa640 1 -- 172.21.15.110:0/4042580915 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e5c0047a0 con 0x560af784d840 2024-04-25T00:08:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.540+0000 7f5e517fa640 1 -- 172.21.15.110:0/4042580915 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c00fa60 con 0x560af784d840 2024-04-25T00:08:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 -- 172.21.15.110:0/4042580915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784d840 msgr2=0x560af7852340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784d840 0x560af7852340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0115c0 tx=0x7f5e5c011a80 comp rx=0 tx=0).stop 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 -- 172.21.15.110:0/4042580915 shutdown_connections 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af78529b0 0x560af7854e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784d840 0x560af7852340 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 --2- 172.21.15.110:0/4042580915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af784d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 -- 172.21.15.110:0/4042580915 >> 172.21.15.110:0/4042580915 conn(0x560af78484c0 msgr2=0x560af78488c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 -- 172.21.15.110:0/4042580915 shutdown_connections 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.541+0000 7f5e74178740 1 -- 172.21.15.110:0/4042580915 wait complete. 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 1 -- start start 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 1 Processor -- start 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784ce50 0x560af78572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af7857810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af78529b0 0x560af785e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560af7855960 con 0x560af784d840 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560af776cb50 con 0x560af784ce50 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560af7855ae0 con 0x560af78529b0 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e74178740 10 client.0 _ll_register_callbacks cb 0x560af77a7ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e6affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af7857810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e6a7fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784ce50 0x560af78572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e6a7fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784ce50 0x560af78572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:38394/0 (socket says 172.21.15.110:38394) 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.542+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/1235733996 learned_addr learned my addr 172.21.15.110:0/1235733996 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af78529b0 0x560af785e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/1235733996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af78529b0 msgr2=0x560af785e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af78529b0 0x560af785e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/1235733996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 msgr2=0x560af7857810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af7857810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/1235733996 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af7591620 con 0x560af784ce50 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784ce50 0x560af78572d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009250 tx=0x7f5e60009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8292b902d57233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1235733996 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60037040 con 0x560af784ce50 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e74178740 1 -- 172.21.15.110:0/1235733996 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e5c0047a0 con 0x560af784ce50 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1235733996 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5e600035f0 con 0x560af784ce50 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.543+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1235733996 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5e600037f0 con 0x560af784ce50 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e74178740 10 client.25340 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e74178740 1 -- 172.21.15.110:0/1235733996 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560af7861c80 con 0x560af784ce50 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1235733996 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e5c0047a0 con 0x560af784ce50 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1235733996 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5e6003d040 con 0x560af784ce50 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e74178740 10 client.25340 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e74178740 10 client.25340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e74178740 20 client.25340 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e74178740 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560af78632a0 0x560af78656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e74178740 4 client.25340 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.544+0000 7f5e74178740 1 -- 172.21.15.110:0/1235733996 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af77be1f0 con 0x560af78632a0 2024-04-25T00:08:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.545+0000 7f5e6affd640 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560af78632a0 0x560af78656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.545+0000 7f5e6affd640 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560af78632a0 0x560af78656d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69432a3fcc0a6912 server_cookie=61d2886282a7808f in_seq=0 out_seq=0 2024-04-25T00:08:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.545+0000 7f5e4eff5640 10 client.25340 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1235733996 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560af77be1f0 con 0x560af78632a0 2024-04-25T00:08:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e4eff5640 10 client.25340 handle_command_reply: tid=0 2024-04-25T00:08:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 client.25340 shutdown 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 -- 172.21.15.110:0/1235733996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784ce50 msgr2=0x560af78572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784ce50 0x560af78572d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009250 tx=0x7f5e60009470 comp rx=0 tx=0).stop 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 -- 172.21.15.110:0/1235733996 shutdown_connections 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560af78632a0 0x560af78656d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af78529b0 0x560af785e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e4eff5640 0 client.25340 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af7857810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 --2- 172.21.15.110:0/1235733996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af784ce50 0x560af78572d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 -- 172.21.15.110:0/1235733996 >> 172.21.15.110:0/1235733996 conn(0x560af78484c0 msgr2=0x560af78475c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 -- 172.21.15.110:0/1235733996 shutdown_connections 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 1 -- 172.21.15.110:0/1235733996 wait complete. 2024-04-25T00:08:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.546+0000 7f5e74178740 20 client.25340 trim_cache size 0 max 16384 2024-04-25T00:08:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e74178740 1 Processor -- start 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e74178740 1 -- start start 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e74178740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784ce50 0x560af784b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e74178740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784d840 0x560af785e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e74178740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 0x560af784b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e74178740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1a90a0 con 0x560af784ce50 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e74178740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1a6970 con 0x560af78529b0 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e74178740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1a47c0 con 0x560af784d840 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e6a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784ce50 0x560af784b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e6a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784ce50 0x560af784b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58876/0 (socket says 172.21.15.110:58876) 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e6affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784d840 0x560af785e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.570+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/3042789544 learned_addr learned my addr 172.21.15.110:0/3042789544 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.571+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/3042789544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 0x560af784b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.571+0000 7f5e6affd640 1 -- 172.21.15.110:0/3042789544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 msgr2=0x560af784b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.571+0000 7f5e6affd640 1 --2- 172.21.15.110:0/3042789544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 0x560af784b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.571+0000 7f5e6affd640 1 -- 172.21.15.110:0/3042789544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784ce50 msgr2=0x560af784b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.571+0000 7f5e6affd640 1 --2- 172.21.15.110:0/3042789544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784ce50 0x560af784b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.571+0000 7f5e6affd640 1 -- 172.21.15.110:0/3042789544 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af7851f80 con 0x560af784d840 2024-04-25T00:08:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.571+0000 7f5e6affd640 1 --2- 172.21.15.110:0/3042789544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784d840 0x560af785e810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x560af7849430 tx=0x7f5e5c004eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b0d8bdc007dc80f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.571+0000 7f5e50ff9640 1 -- 172.21.15.110:0/3042789544 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c01d040 con 0x560af784d840 2024-04-25T00:08:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.572+0000 7f5e50ff9640 1 -- 172.21.15.110:0/3042789544 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e5c005d30 con 0x560af784d840 2024-04-25T00:08:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.572+0000 7f5e50ff9640 1 -- 172.21.15.110:0/3042789544 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c00faa0 con 0x560af784d840 2024-04-25T00:08:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.572+0000 7f5e74178740 1 -- 172.21.15.110:0/3042789544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784d840 msgr2=0x560af785e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.572+0000 7f5e74178740 1 --2- 172.21.15.110:0/3042789544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784d840 0x560af785e810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x560af7849430 tx=0x7f5e5c004eb0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 -- 172.21.15.110:0/3042789544 shutdown_connections 2024-04-25T00:08:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 --2- 172.21.15.110:0/3042789544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 0x560af784b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 --2- 172.21.15.110:0/3042789544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784d840 0x560af785e810 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 --2- 172.21.15.110:0/3042789544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784ce50 0x560af784b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 -- 172.21.15.110:0/3042789544 >> 172.21.15.110:0/3042789544 conn(0x560af78484c0 msgr2=0x560af78488a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 -- 172.21.15.110:0/3042789544 shutdown_connections 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 -- 172.21.15.110:0/3042789544 wait complete. 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 -- start start 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.573+0000 7f5e74178740 1 Processor -- start 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e74178740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af785bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e74178740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af785c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e74178740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 0x560af785d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e74178740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1a4e30 con 0x560af784d840 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e74178740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1a62a0 con 0x560af78529b0 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e74178740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1a5500 con 0x560af784ce50 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e74178740 10 client.0 _ll_register_callbacks cb 0x7f5e64009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af785bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af785bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:40926/0 (socket says 172.21.15.110:40926) 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af785c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/1985934466 learned_addr learned my addr 172.21.15.110:0/1985934466 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 0x560af785d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/1985934466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 msgr2=0x560af785d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 0x560af785d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/1985934466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 msgr2=0x560af785c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af785c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6a7fc640 1 -- 172.21.15.110:0/1985934466 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af785d8f0 con 0x560af784ce50 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.574+0000 7f5e6affd640 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af785c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e6a7fc640 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af785bdb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e60030dd0 tx=0x7f5e60000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22128be956f55299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:08:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1985934466 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60037040 con 0x560af784ce50 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e74178740 1 -- 172.21.15.110:0/1985934466 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e5c005d30 con 0x560af784ce50 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1985934466 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5e60005030 con 0x560af784ce50 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1985934466 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5e60000e60 con 0x560af784ce50 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e74178740 10 client.25318 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e74178740 1 -- 172.21.15.110:0/1985934466 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560af78c9850 con 0x560af784ce50 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.575+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1985934466 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e5c005d30 con 0x560af784ce50 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.576+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1985934466 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5e6003d040 con 0x560af784ce50 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.576+0000 7f5e74178740 10 client.25318 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.576+0000 7f5e74178740 10 client.25318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.576+0000 7f5e74178740 20 client.25318 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.576+0000 7f5e74178740 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560af78632a0 0x560af78cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.576+0000 7f5e74178740 4 client.25318 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:08:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.576+0000 7f5e74178740 1 -- 172.21.15.110:0/1985934466 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560af77b8540 con 0x560af78632a0 2024-04-25T00:08:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.576+0000 7f5e6affd640 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560af78632a0 0x560af78cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.577+0000 7f5e6affd640 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560af78632a0 0x560af78cc0f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d94c5f8ee6a0c5 server_cookie=c936befaad90c2cb in_seq=0 out_seq=0 2024-04-25T00:08:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.577+0000 7f5e4eff5640 10 client.25318 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 310.0 s, 779.0 tps, lat 128.283 ms stddev 79.232 2024-04-25T00:08:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.593+0000 7f5e4eff5640 1 -- 172.21.15.110:0/1985934466 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560af77b8540 con 0x560af78632a0 2024-04-25T00:08:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.593+0000 7f5e4eff5640 10 client.25318 handle_command_reply: tid=0 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.593+0000 7f5e74178740 1 client.25318 shutdown 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 -- 172.21.15.110:0/1985934466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 msgr2=0x560af785bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af785bdb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e60030dd0 tx=0x7f5e60000bc0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 -- 172.21.15.110:0/1985934466 shutdown_connections 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x560af78632a0 0x560af78cc0f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e4eff5640 0 client.25318 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560af78529b0 0x560af785d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af784d840 0x560af785c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 --2- 172.21.15.110:0/1985934466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560af784ce50 0x560af785bdb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 -- 172.21.15.110:0/1985934466 >> 172.21.15.110:0/1985934466 conn(0x560af78484c0 msgr2=0x560af784b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 -- 172.21.15.110:0/1985934466 shutdown_connections 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 1 -- 172.21.15.110:0/1985934466 wait complete. 2024-04-25T00:08:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.594+0000 7f5e74178740 20 client.25318 trim_cache size 0 max 16384 2024-04-25T00:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:08:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 -- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5e540a2c30 msgr2=0x7f5e540a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5e540a2c30 0x7f5e540a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5e640a8170 tx=0x7f5e64025000 comp rx=0 tx=0).stop 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 -- 172.21.15.110:0/4089769598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6c102250 msgr2=0x7f5e6c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6c102250 0x7f5e6c19ef30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400b670 tx=0x7f5e6400bb30 comp rx=0 tx=0).stop 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 -- 172.21.15.110:0/4089769598 shutdown_connections 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5e540a2c30 0x7f5e540a50f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6c102250 0x7f5e6c19ef30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e6c101890 0x7f5e6c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 --2- 172.21.15.110:0/4089769598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e6c1006a0 0x7f5e6c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 -- 172.21.15.110:0/4089769598 >> 172.21.15.110:0/4089769598 conn(0x7f5e6c0fbc10 msgr2=0x7f5e6c104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 -- 172.21.15.110:0/4089769598 shutdown_connections 2024-04-25T00:08:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:16.595+0000 7f5e71390640 1 -- 172.21.15.110:0/4089769598 wait complete. 2024-04-25T00:08:17.988 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:17 smithi110 ceph-mon[29187]: pgmap v538: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 910 op/s 2024-04-25T00:08:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:17 smithi188 ceph-mon[33285]: pgmap v538: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 910 op/s 2024-04-25T00:08:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:17 smithi173 ceph-mon[32402]: pgmap v538: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 910 op/s 2024-04-25T00:08:20.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:19 smithi188 ceph-mon[33285]: pgmap v539: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-04-25T00:08:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:19 smithi110 ceph-mon[29187]: pgmap v539: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-04-25T00:08:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:19 smithi173 ceph-mon[32402]: pgmap v539: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-04-25T00:08:21.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 315.0 s, 804.6 tps, lat 124.606 ms stddev 82.603 2024-04-25T00:08:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:21 smithi188 ceph-mon[33285]: pgmap v540: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 905 op/s 2024-04-25T00:08:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:21 smithi110 ceph-mon[29187]: pgmap v540: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 905 op/s 2024-04-25T00:08:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:21 smithi173 ceph-mon[32402]: pgmap v540: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 905 op/s 2024-04-25T00:08:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:23 smithi188 ceph-mon[33285]: pgmap v541: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:08:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:23 smithi110 ceph-mon[29187]: pgmap v541: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:08:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:23 smithi173 ceph-mon[32402]: pgmap v541: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:08:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:25 smithi188 ceph-mon[33285]: pgmap v542: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:08:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:25 smithi110 ceph-mon[29187]: pgmap v542: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:08:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:25 smithi173 ceph-mon[32402]: pgmap v542: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:08:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 320.0 s, 757.8 tps, lat 131.668 ms stddev 78.422 2024-04-25T00:08:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:27 smithi188 ceph-mon[33285]: pgmap v543: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 891 op/s 2024-04-25T00:08:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:27 smithi110 ceph-mon[29187]: pgmap v543: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 891 op/s 2024-04-25T00:08:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:27 smithi173 ceph-mon[32402]: pgmap v543: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 891 op/s 2024-04-25T00:08:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.712+0000 7f281c363640 1 -- 172.21.15.110:0/2800543246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2814102250 msgr2=0x7f2814108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.712+0000 7f281c363640 1 --2- 172.21.15.110:0/2800543246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2814102250 0x7f2814108ae0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f280c009900 tx=0x7f280c02a410 comp rx=0 tx=0).stop 2024-04-25T00:08:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.713+0000 7f281c363640 1 -- 172.21.15.110:0/2800543246 shutdown_connections 2024-04-25T00:08:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.713+0000 7f281c363640 1 --2- 172.21.15.110:0/2800543246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2814102250 0x7f2814108ae0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.713+0000 7f281c363640 1 --2- 172.21.15.110:0/2800543246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2814101890 0x7f2814101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.713+0000 7f281c363640 1 --2- 172.21.15.110:0/2800543246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28141006a0 0x7f2814100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.713+0000 7f281c363640 1 -- 172.21.15.110:0/2800543246 >> 172.21.15.110:0/2800543246 conn(0x7f28140fbbf0 msgr2=0x7f28140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.713+0000 7f281c363640 1 -- 172.21.15.110:0/2800543246 shutdown_connections 2024-04-25T00:08:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.713+0000 7f281c363640 1 -- 172.21.15.110:0/2800543246 wait complete. 2024-04-25T00:08:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.715+0000 7f281c363640 1 Processor -- start 2024-04-25T00:08:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.715+0000 7f281c363640 1 -- start start 2024-04-25T00:08:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f281c363640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28141006a0 0x7f2814199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f281c363640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2814101890 0x7f281419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f281c363640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2814102250 0x7f28141a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f281c363640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f281406a540 con 0x7f2814102250 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f281c363640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f281406a3c0 con 0x7f28141006a0 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f281c363640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f281406a6c0 con 0x7f2814101890 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f28198fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2814101890 0x7f281419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f28198fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2814101890 0x7f281419a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:33976/0 (socket says 172.21.15.110:33976) 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f28198fe640 1 -- 172.21.15.110:0/2440571561 learned_addr learned my addr 172.21.15.110:0/2440571561 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f281a900640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2814102250 0x7f28141a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f281a0ff640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28141006a0 0x7f2814199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f28198fe640 1 -- 172.21.15.110:0/2440571561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28141006a0 msgr2=0x7f2814199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f28198fe640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28141006a0 0x7f2814199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f28198fe640 1 -- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2814102250 msgr2=0x7f28141a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f28198fe640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2814102250 0x7f28141a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.716+0000 7f28198fe640 1 -- 172.21.15.110:0/2440571561 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28141a1700 con 0x7f2814101890 2024-04-25T00:08:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.717+0000 7f28198fe640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2814101890 0x7f281419a020 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f280400d950 tx=0x7f280400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.717+0000 7f280b7fe640 1 -- 172.21.15.110:0/2440571561 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804017070 con 0x7f2814101890 2024-04-25T00:08:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.717+0000 7f280b7fe640 1 -- 172.21.15.110:0/2440571561 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2804009c30 con 0x7f2814101890 2024-04-25T00:08:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.717+0000 7f281c363640 1 -- 172.21.15.110:0/2440571561 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28141a19f0 con 0x7f2814101890 2024-04-25T00:08:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.717+0000 7f280b7fe640 1 -- 172.21.15.110:0/2440571561 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280400f620 con 0x7f2814101890 2024-04-25T00:08:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.717+0000 7f281c363640 1 -- 172.21.15.110:0/2440571561 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141a1f90 con 0x7f2814101890 2024-04-25T00:08:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.720+0000 7f280b7fe640 1 -- 172.21.15.110:0/2440571561 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f28040109a0 con 0x7f2814101890 2024-04-25T00:08:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.720+0000 7f281f06a740 1 Processor -- start 2024-04-25T00:08:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.720+0000 7f281f06a740 1 -- start start 2024-04-25T00:08:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281f06a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff76270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281f06a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff76840 0x5574fff7b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281f06a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff7b9b0 0x5574fff7de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281f06a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574ffed9cb0 con 0x5574fff75e50 2024-04-25T00:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281f06a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5574ffe825b0 con 0x5574fff7b9b0 2024-04-25T00:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281f06a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5574ffed5c70 con 0x5574fff76840 2024-04-25T00:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281a900640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff76270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281a900640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff76270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34814/0 (socket says 172.21.15.110:34814) 2024-04-25T00:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281a900640 1 -- 172.21.15.110:0/2415916180 learned_addr learned my addr 172.21.15.110:0/2415916180 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f28198fe640 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff7b9b0 0x5574fff7de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281a0ff640 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff76840 0x5574fff7b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281a900640 1 -- 172.21.15.110:0/2415916180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff76840 msgr2=0x5574fff7b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.721+0000 7f281a900640 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff76840 0x5574fff7b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f281a900640 1 -- 172.21.15.110:0/2415916180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff7b9b0 msgr2=0x5574fff7de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f281a900640 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff7b9b0 0x5574fff7de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f281a900640 1 -- 172.21.15.110:0/2415916180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ffbee210 con 0x5574fff75e50 2024-04-25T00:08:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f280b7fe640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f28000a2cb0 0x7f28000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f281a900640 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff76270 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f280c009230 tx=0x7f280c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4558e8410a8db385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f280b7fe640 1 -- 172.21.15.110:0/2440571561 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f28040ceb30 con 0x7f2814101890 2024-04-25T00:08:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f2808bfe640 1 -- 172.21.15.110:0/2415916180 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c035030 con 0x5574fff75e50 2024-04-25T00:08:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f2808bfe640 1 -- 172.21.15.110:0/2415916180 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c040070 con 0x5574fff75e50 2024-04-25T00:08:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f281a0ff640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f28000a2cb0 0x7f28000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.722+0000 7f2808bfe640 1 -- 172.21.15.110:0/2415916180 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c003a30 con 0x5574fff75e50 2024-04-25T00:08:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 -- 172.21.15.110:0/2415916180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 msgr2=0x5574fff76270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff76270 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f280c009230 tx=0x7f280c031af0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281a0ff640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f28000a2cb0 0x7f28000a5170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5574fff76c40 tx=0x7f2810009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 -- 172.21.15.110:0/2415916180 shutdown_connections 2024-04-25T00:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff7b9b0 0x5574fff7de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff76840 0x5574fff7b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 --2- 172.21.15.110:0/2415916180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff76270 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 -- 172.21.15.110:0/2415916180 >> 172.21.15.110:0/2415916180 conn(0x5574fff714c0 msgr2=0x5574fff718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 -- 172.21.15.110:0/2415916180 shutdown_connections 2024-04-25T00:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.723+0000 7f281f06a740 1 -- 172.21.15.110:0/2415916180 wait complete. 2024-04-25T00:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281f06a740 1 -- start start 2024-04-25T00:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281f06a740 1 Processor -- start 2024-04-25T00:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281f06a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff75e50 0x5574fff88ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281f06a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff76840 0x5574fff89010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281f06a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574fff90050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281f06a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574fff7e950 con 0x5574fff76840 2024-04-25T00:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281f06a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5574ffe95b50 con 0x5574fff75e50 2024-04-25T00:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281f06a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5574fff7ead0 con 0x5574fff7b9b0 2024-04-25T00:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281a0ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff75e50 0x5574fff88ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281a0ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff75e50 0x5574fff88ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39080/0 (socket says 172.21.15.110:39080) 2024-04-25T00:08:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281a0ff640 1 -- 172.21.15.110:0/158663062 learned_addr learned my addr 172.21.15.110:0/158663062 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.724+0000 7f281a900640 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574fff90050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f28198fe640 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff76840 0x5574fff89010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f281f06a740 10 client.0 _ll_register_callbacks cb 0x5574ffed0ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f281a0ff640 1 -- 172.21.15.110:0/158663062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 msgr2=0x5574fff90050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f281a0ff640 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574fff90050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f281a0ff640 1 -- 172.21.15.110:0/158663062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff76840 msgr2=0x5574fff89010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f281a0ff640 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff76840 0x5574fff89010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f281a0ff640 1 -- 172.21.15.110:0/158663062 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ffcba620 con 0x5574fff75e50 2024-04-25T00:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f281a900640 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574fff90050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f281a0ff640 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff75e50 0x5574fff88ad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2810009990 tx=0x7f2810004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c3bda792a453a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f27fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.725+0000 7f27fdffb640 1 -- 172.21.15.110:0/158663062 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281002c060 con 0x5574fff75e50 2024-04-25T00:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.726+0000 7f281f06a740 1 -- 172.21.15.110:0/158663062 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280c040070 con 0x5574fff75e50 2024-04-25T00:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.726+0000 7f27fdffb640 1 -- 172.21.15.110:0/158663062 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2810018030 con 0x5574fff75e50 2024-04-25T00:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.726+0000 7f27fdffb640 1 -- 172.21.15.110:0/158663062 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f281000f8d0 con 0x5574fff75e50 2024-04-25T00:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f281f06a740 10 client.25358 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f281f06a740 1 -- 172.21.15.110:0/158663062 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5574fff93460 con 0x5574fff75e50 2024-04-25T00:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f27fdffb640 1 -- 172.21.15.110:0/158663062 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c040070 con 0x5574fff75e50 2024-04-25T00:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f27fdffb640 1 -- 172.21.15.110:0/158663062 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f281001fb30 con 0x5574fff75e50 2024-04-25T00:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f281f06a740 10 client.25358 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f281f06a740 10 client.25358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f281f06a740 20 client.25358 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f281f06a740 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574fff94f80 0x5574fff97440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f281f06a740 4 client.25358 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f281f06a740 1 -- 172.21.15.110:0/158663062 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574ffee71f0 con 0x5574fff94f80 2024-04-25T00:08:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.727+0000 7f28198fe640 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574fff94f80 0x5574fff97440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.728+0000 7f27fdffb640 10 client.25358 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.728+0000 7f28198fe640 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574fff94f80 0x5574fff97440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06291756cefee95 server_cookie=4869dbcd9a1502c7 in_seq=0 out_seq=0 2024-04-25T00:08:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f27fdffb640 1 -- 172.21.15.110:0/158663062 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574ffee71f0 con 0x5574fff94f80 2024-04-25T00:08:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f27fdffb640 10 client.25358 handle_command_reply: tid=0 2024-04-25T00:08:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 client.25358 shutdown 2024-04-25T00:08:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 -- 172.21.15.110:0/158663062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff75e50 msgr2=0x5574fff88ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff75e50 0x5574fff88ad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2810009990 tx=0x7f2810004ab0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 -- 172.21.15.110:0/158663062 shutdown_connections 2024-04-25T00:08:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574fff94f80 0x5574fff97440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574fff90050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff76840 0x5574fff89010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f27fdffb640 0 client.25358 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 --2- 172.21.15.110:0/158663062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff75e50 0x5574fff88ad0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 -- 172.21.15.110:0/158663062 >> 172.21.15.110:0/158663062 conn(0x5574fff714c0 msgr2=0x5574fff705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.729+0000 7f281f06a740 1 -- 172.21.15.110:0/158663062 shutdown_connections 2024-04-25T00:08:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.730+0000 7f281f06a740 1 -- 172.21.15.110:0/158663062 wait complete. 2024-04-25T00:08:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.730+0000 7f281f06a740 20 client.25358 trim_cache size 0 max 16384 2024-04-25T00:08:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281f06a740 1 Processor -- start 2024-04-25T00:08:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281f06a740 1 -- start start 2024-04-25T00:08:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281f06a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff75e50 0x5574fff886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281f06a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff90010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281f06a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff7b9b0 0x5574fff88c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281f06a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a9130 con 0x5574fff7b9b0 2024-04-25T00:08:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281f06a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a6a00 con 0x5574fff76840 2024-04-25T00:08:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281f06a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a48b0 con 0x5574fff75e50 2024-04-25T00:08:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281a0ff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff75e50 0x5574fff886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f28198fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff90010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f28198fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff90010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39086/0 (socket says 172.21.15.110:39086) 2024-04-25T00:08:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f28198fe640 1 -- 172.21.15.110:0/1227823542 learned_addr learned my addr 172.21.15.110:0/1227823542 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281a0ff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff75e50 0x5574fff886e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:33990/0 (socket says 172.21.15.110:33990) 2024-04-25T00:08:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f281a900640 1 --2- 172.21.15.110:0/1227823542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff7b9b0 0x5574fff88c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f28198fe640 1 -- 172.21.15.110:0/1227823542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff75e50 msgr2=0x5574fff886e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f28198fe640 1 --2- 172.21.15.110:0/1227823542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff75e50 0x5574fff886e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f28198fe640 1 -- 172.21.15.110:0/1227823542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff7b9b0 msgr2=0x5574fff88c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f28198fe640 1 --2- 172.21.15.110:0/1227823542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff7b9b0 0x5574fff88c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.760+0000 7f28198fe640 1 -- 172.21.15.110:0/1227823542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fff7af70 con 0x5574fff76840 2024-04-25T00:08:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.761+0000 7f28198fe640 1 --2- 172.21.15.110:0/1227823542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff90010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2804097e40 tx=0x7f2804097040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=165b1fd897a356aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.761+0000 7f27fffff640 1 -- 172.21.15.110:0/1227823542 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804020040 con 0x5574fff76840 2024-04-25T00:08:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.761+0000 7f27fffff640 1 -- 172.21.15.110:0/1227823542 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2804009de0 con 0x5574fff76840 2024-04-25T00:08:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.761+0000 7f27fffff640 1 -- 172.21.15.110:0/1227823542 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804097ae0 con 0x5574fff76840 2024-04-25T00:08:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 -- 172.21.15.110:0/1227823542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 msgr2=0x5574fff90010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 --2- 172.21.15.110:0/1227823542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff90010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2804097e40 tx=0x7f2804097040 comp rx=0 tx=0).stop 2024-04-25T00:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 -- 172.21.15.110:0/1227823542 shutdown_connections 2024-04-25T00:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 --2- 172.21.15.110:0/1227823542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff7b9b0 0x5574fff88c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 --2- 172.21.15.110:0/1227823542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff90010 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 --2- 172.21.15.110:0/1227823542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff75e50 0x5574fff886e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 -- 172.21.15.110:0/1227823542 >> 172.21.15.110:0/1227823542 conn(0x5574fff714c0 msgr2=0x5574fff718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 -- 172.21.15.110:0/1227823542 shutdown_connections 2024-04-25T00:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 -- 172.21.15.110:0/1227823542 wait complete. 2024-04-25T00:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 -- start start 2024-04-25T00:08:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 Processor -- start 2024-04-25T00:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.762+0000 7f281f06a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff8cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281f06a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff8d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281f06a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574ffff95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281f06a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a4f00 con 0x5574fff75e50 2024-04-25T00:08:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281f06a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a6330 con 0x5574fff76840 2024-04-25T00:08:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281f06a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a5590 con 0x5574fff7b9b0 2024-04-25T00:08:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281f06a740 10 client.0 _ll_register_callbacks cb 0x7f2810003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281a0ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff8cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281a0ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff8cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34850/0 (socket says 172.21.15.110:34850) 2024-04-25T00:08:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f28198fe640 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff8d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281a900640 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574ffff95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281a0ff640 1 -- 172.21.15.110:0/2801094147 learned_addr learned my addr 172.21.15.110:0/2801094147 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f28198fe640 1 -- 172.21.15.110:0/2801094147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 msgr2=0x5574ffff95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f28198fe640 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574ffff95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f28198fe640 1 -- 172.21.15.110:0/2801094147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 msgr2=0x5574fff8cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f28198fe640 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff8cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f28198fe640 1 -- 172.21.15.110:0/2801094147 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fff90280 con 0x5574fff76840 2024-04-25T00:08:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f28198fe640 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff8d1b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2804000c00 tx=0x7f2804009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104accfa7c83c246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f27fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f27fdffb640 1 -- 172.21.15.110:0/2801094147 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804020040 con 0x5574fff76840 2024-04-25T00:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.763+0000 7f281f06a740 1 -- 172.21.15.110:0/2801094147 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2804009de0 con 0x5574fff76840 2024-04-25T00:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f27fdffb640 1 -- 172.21.15.110:0/2801094147 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2804097ae0 con 0x5574fff76840 2024-04-25T00:08:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f27fdffb640 1 -- 172.21.15.110:0/2801094147 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f280409f770 con 0x5574fff76840 2024-04-25T00:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281f06a740 10 client.25364 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281f06a740 1 -- 172.21.15.110:0/2801094147 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5574ffffab30 con 0x5574fff76840 2024-04-25T00:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f27fdffb640 1 -- 172.21.15.110:0/2801094147 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2804009de0 con 0x5574fff76840 2024-04-25T00:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f27fdffb640 1 -- 172.21.15.110:0/2801094147 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f28040ccbb0 con 0x5574fff76840 2024-04-25T00:08:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281f06a740 10 client.25364 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281f06a740 10 client.25364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281f06a740 20 client.25364 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281f06a740 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574ffffb480 0x5574ffffd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281f06a740 4 client.25364 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:08:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281f06a740 1 -- 172.21.15.110:0/2801094147 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574ffee1540 con 0x5574ffffb480 2024-04-25T00:08:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.764+0000 7f281a0ff640 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574ffffb480 0x5574ffffd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.765+0000 7f281a0ff640 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574ffffb480 0x5574ffffd8a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d237bad3c8b9e567 server_cookie=e9768ad9a55530fb in_seq=0 out_seq=0 2024-04-25T00:08:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.765+0000 7f27fdffb640 10 client.25364 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.785+0000 7f27fdffb640 1 -- 172.21.15.110:0/2801094147 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574ffee1540 con 0x5574ffffb480 2024-04-25T00:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.785+0000 7f27fdffb640 10 client.25364 handle_command_reply: tid=0 2024-04-25T00:08:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.785+0000 7f281f06a740 1 client.25364 shutdown 2024-04-25T00:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.785+0000 7f281f06a740 1 -- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 msgr2=0x5574fff8d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff8d1b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2804000c00 tx=0x7f2804009740 comp rx=0 tx=0).stop 2024-04-25T00:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 -- 172.21.15.110:0/2801094147 shutdown_connections 2024-04-25T00:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574ffffb480 0x5574ffffd8a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574fff7b9b0 0x5574ffff95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574fff76840 0x5574fff8d1b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 --2- 172.21.15.110:0/2801094147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574fff75e50 0x5574fff8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 -- 172.21.15.110:0/2801094147 >> 172.21.15.110:0/2801094147 conn(0x5574fff714c0 msgr2=0x5574fff88650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f27fdffb640 0 client.25364 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 -- 172.21.15.110:0/2801094147 shutdown_connections 2024-04-25T00:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 1 -- 172.21.15.110:0/2801094147 wait complete. 2024-04-25T00:08:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.786+0000 7f281f06a740 20 client.25364 trim_cache size 0 max 16384 2024-04-25T00:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:08:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:08:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 -- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f28000a2cb0 msgr2=0x7f28000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f28000a2cb0 0x7f28000a5170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5574fff76c40 tx=0x7f2810009500 comp rx=0 tx=0).stop 2024-04-25T00:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 -- 172.21.15.110:0/2440571561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2814101890 msgr2=0x7f281419a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2814101890 0x7f281419a020 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f280400d950 tx=0x7f280400de10 comp rx=0 tx=0).stop 2024-04-25T00:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 -- 172.21.15.110:0/2440571561 shutdown_connections 2024-04-25T00:08:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f28000a2cb0 0x7f28000a5170 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2814102250 0x7f28141a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2814101890 0x7f281419a020 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 --2- 172.21.15.110:0/2440571561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28141006a0 0x7f2814199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 -- 172.21.15.110:0/2440571561 >> 172.21.15.110:0/2440571561 conn(0x7f28140fbbf0 msgr2=0x7f281410b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 -- 172.21.15.110:0/2440571561 shutdown_connections 2024-04-25T00:08:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:29.787+0000 7f281c363640 1 -- 172.21.15.110:0/2440571561 wait complete. 2024-04-25T00:08:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:29 smithi188 ceph-mon[33285]: pgmap v544: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-04-25T00:08:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:29 smithi110 ceph-mon[29187]: pgmap v544: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-04-25T00:08:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:29 smithi173 ceph-mon[32402]: pgmap v544: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-04-25T00:08:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 325.0 s, 836.6 tps, lat 119.628 ms stddev 14.024 2024-04-25T00:08:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:31 smithi188 ceph-mon[33285]: pgmap v545: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 901 op/s 2024-04-25T00:08:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:31 smithi110 ceph-mon[29187]: pgmap v545: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 901 op/s 2024-04-25T00:08:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:31 smithi173 ceph-mon[32402]: pgmap v545: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 MiB/s wr, 901 op/s 2024-04-25T00:08:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:33 smithi188 ceph-mon[33285]: pgmap v546: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-04-25T00:08:34.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:33 smithi110 ceph-mon[29187]: pgmap v546: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-04-25T00:08:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:33 smithi173 ceph-mon[32402]: pgmap v546: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-04-25T00:08:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:34 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:35.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:34 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:35.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:34 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:35 smithi188 ceph-mon[33285]: pgmap v547: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 972 op/s 2024-04-25T00:08:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:08:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:35 smithi110 ceph-mon[29187]: pgmap v547: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 972 op/s 2024-04-25T00:08:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:36.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:36.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:08:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:35 smithi173 ceph-mon[32402]: pgmap v547: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 972 op/s 2024-04-25T00:08:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:36.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:36.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:36.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:08:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 330.0 s, 786.8 tps, lat 127.087 ms stddev 81.916 2024-04-25T00:08:37.002 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:36 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-04-25T00:08:37.002 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:36 smithi173 ceph-mon[32402]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T00:08:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:36 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-04-25T00:08:37.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:36 smithi188 ceph-mon[33285]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T00:08:37.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:36 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-04-25T00:08:37.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:36 smithi110 ceph-mon[29187]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T00:08:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:37 smithi188 ceph-mon[33285]: pgmap v548: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 927 op/s 2024-04-25T00:08:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:37 smithi110 ceph-mon[29187]: pgmap v548: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 927 op/s 2024-04-25T00:08:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:37 smithi173 ceph-mon[32402]: pgmap v548: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 927 op/s 2024-04-25T00:08:40.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:39 smithi188 ceph-mon[33285]: pgmap v549: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2024-04-25T00:08:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:39 smithi110 ceph-mon[29187]: pgmap v549: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2024-04-25T00:08:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:39 smithi173 ceph-mon[32402]: pgmap v549: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2024-04-25T00:08:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 335.0 s, 788.8 tps, lat 126.758 ms stddev 78.370 2024-04-25T00:08:42.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:41 smithi188 ceph-mon[33285]: pgmap v550: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 934 op/s 2024-04-25T00:08:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:41 smithi110 ceph-mon[29187]: pgmap v550: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 934 op/s 2024-04-25T00:08:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:41 smithi173 ceph-mon[32402]: pgmap v550: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 934 op/s 2024-04-25T00:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 -- 172.21.15.110:0/3490984358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c0ffe30 msgr2=0x7f446c100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 --2- 172.21.15.110:0/3490984358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c0ffe30 0x7f446c100250 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4464009ca0 tx=0x7f446402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 -- 172.21.15.110:0/3490984358 shutdown_connections 2024-04-25T00:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 --2- 172.21.15.110:0/3490984358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c101940 0x7f446c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 --2- 172.21.15.110:0/3490984358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c100f80 0x7f446c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 --2- 172.21.15.110:0/3490984358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c0ffe30 0x7f446c100250 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 -- 172.21.15.110:0/3490984358 >> 172.21.15.110:0/3490984358 conn(0x7f446c0fb3a0 msgr2=0x7f446c0fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 -- 172.21.15.110:0/3490984358 shutdown_connections 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.884+0000 7f4470e12640 1 -- 172.21.15.110:0/3490984358 wait complete. 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4470e12640 1 Processor -- start 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4470e12640 1 -- start start 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4470e12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c0ffe30 0x7f446c1097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4470e12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c100f80 0x7f446c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4470e12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c101940 0x7f446c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4470e12640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f446c10c280 con 0x7f446c100f80 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4470e12640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f446c10c100 con 0x7f446c101940 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4470e12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f446c10c400 con 0x7f446c0ffe30 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4469d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c100f80 0x7f446c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4469d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c100f80 0x7f446c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51438/0 (socket says 172.21.15.110:51438) 2024-04-25T00:08:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f4469d9b640 1 -- 172.21.15.110:0/724980296 learned_addr learned my addr 172.21.15.110:0/724980296 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f446ad9d640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c101940 0x7f446c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.885+0000 7f446a59c640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c0ffe30 0x7f446c1097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f4469d9b640 1 -- 172.21.15.110:0/724980296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c0ffe30 msgr2=0x7f446c1097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f4469d9b640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c0ffe30 0x7f446c1097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f4469d9b640 1 -- 172.21.15.110:0/724980296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c101940 msgr2=0x7f446c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f4469d9b640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c101940 0x7f446c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f4469d9b640 1 -- 172.21.15.110:0/724980296 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c19f650 con 0x7f446c100f80 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f446a59c640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c0ffe30 0x7f446c1097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f4469d9b640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c100f80 0x7f446c19c720 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f446000b450 tx=0x7f446000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f445b7fe640 1 -- 172.21.15.110:0/724980296 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460015020 con 0x7f446c100f80 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f4470e12640 1 -- 172.21.15.110:0/724980296 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c19f8e0 con 0x7f446c100f80 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f445b7fe640 1 -- 172.21.15.110:0/724980296 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4460011d40 con 0x7f446c100f80 2024-04-25T00:08:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f4470e12640 1 -- 172.21.15.110:0/724980296 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c19fe80 con 0x7f446c100f80 2024-04-25T00:08:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.886+0000 7f445b7fe640 1 -- 172.21.15.110:0/724980296 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446000f340 con 0x7f446c100f80 2024-04-25T00:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.888+0000 7f445b7fe640 1 -- 172.21.15.110:0/724980296 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4460018020 con 0x7f446c100f80 2024-04-25T00:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4473bf9740 1 Processor -- start 2024-04-25T00:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4473bf9740 1 -- start start 2024-04-25T00:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f445b7fe640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44540a2c30 0x7f44540a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1e90270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 0x55fea1e95340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 0x55fea1e97e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4473bf9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fea1df3cb0 con 0x55fea1e8fe50 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4473bf9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fea1d9c5b0 con 0x55fea1e959b0 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4473bf9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fea1defc70 con 0x55fea1e90840 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1e90270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f445b7fe640 1 -- 172.21.15.110:0/724980296 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f44600ceac0 con 0x7f446c100f80 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446ad9d640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44540a2c30 0x7f44540a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1e90270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51442/0 (socket says 172.21.15.110:51442) 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446a59c640 1 -- 172.21.15.110:0/2869421483 learned_addr learned my addr 172.21.15.110:0/2869421483 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446ad9d640 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 0x55fea1e97e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f4469d9b640 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 0x55fea1e95340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446a59c640 1 -- 172.21.15.110:0/2869421483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 msgr2=0x55fea1e95340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446a59c640 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 0x55fea1e95340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446a59c640 1 -- 172.21.15.110:0/2869421483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 msgr2=0x55fea1e97e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446a59c640 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 0x55fea1e97e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.889+0000 7f446a59c640 1 -- 172.21.15.110:0/2869421483 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea1b08210 con 0x55fea1e8fe50 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.890+0000 7f446ad9d640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44540a2c30 0x7f44540a50f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b590 tx=0x7f445c0127c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.890+0000 7f446a59c640 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1e90270 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4464009ca0 tx=0x7f4464009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b0c44e8d8ead35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.890+0000 7f4458ff9640 1 -- 172.21.15.110:0/2869421483 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464031b40 con 0x55fea1e8fe50 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.890+0000 7f4458ff9640 1 -- 172.21.15.110:0/2869421483 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4464036040 con 0x55fea1e8fe50 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.890+0000 7f4458ff9640 1 -- 172.21.15.110:0/2869421483 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464004610 con 0x55fea1e8fe50 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.890+0000 7f4473bf9740 1 -- 172.21.15.110:0/2869421483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 msgr2=0x55fea1e90270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.890+0000 7f4473bf9740 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1e90270 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4464009ca0 tx=0x7f4464009a20 comp rx=0 tx=0).stop 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 -- 172.21.15.110:0/2869421483 shutdown_connections 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 0x55fea1e97e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 0x55fea1e95340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 --2- 172.21.15.110:0/2869421483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1e90270 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 -- 172.21.15.110:0/2869421483 >> 172.21.15.110:0/2869421483 conn(0x55fea1e8b4c0 msgr2=0x55fea1e8b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 -- 172.21.15.110:0/2869421483 shutdown_connections 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 -- 172.21.15.110:0/2869421483 wait complete. 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 -- start start 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 Processor -- start 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e8fe50 0x55fea1e9a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e959b0 0x55fea1ea1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fea1e98900 con 0x55fea1e90840 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fea1dafb50 con 0x55fea1e8fe50 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fea1e98a80 con 0x55fea1e959b0 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4473bf9740 10 client.0 _ll_register_callbacks cb 0x55fea1deaca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4469d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4469d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51448/0 (socket says 172.21.15.110:51448) 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f4469d9b640 1 -- 172.21.15.110:0/3014004506 learned_addr learned my addr 172.21.15.110:0/3014004506 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.891+0000 7f446a59c640 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e8fe50 0x55fea1e9a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f446ad9d640 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e959b0 0x55fea1ea1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4469d9b640 1 -- 172.21.15.110:0/3014004506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e959b0 msgr2=0x55fea1ea1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4469d9b640 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e959b0 0x55fea1ea1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4469d9b640 1 -- 172.21.15.110:0/3014004506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e8fe50 msgr2=0x55fea1e9a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4469d9b640 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e8fe50 0x55fea1e9a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4469d9b640 1 -- 172.21.15.110:0/3014004506 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea1bd4620 con 0x55fea1e90840 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4469d9b640 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4460009fd0 tx=0x7f446009ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d82063110c38beb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4451ffb640 1 -- 172.21.15.110:0/3014004506 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446009f700 con 0x55fea1e90840 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4451ffb640 1 -- 172.21.15.110:0/3014004506 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4460010570 con 0x55fea1e90840 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 1 -- 172.21.15.110:0/3014004506 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4464036040 con 0x55fea1e90840 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4451ffb640 1 -- 172.21.15.110:0/3014004506 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f44600a3fc0 con 0x55fea1e90840 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 10 client.15789 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 1 -- 172.21.15.110:0/3014004506 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fea1ea4ba0 con 0x55fea1e90840 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4451ffb640 1 -- 172.21.15.110:0/3014004506 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4464036040 con 0x55fea1e90840 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4451ffb640 1 -- 172.21.15.110:0/3014004506 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f44600970a0 con 0x55fea1e90840 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 10 client.15789 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 20 client.15789 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55fea1ea6220 0x55fea1ea8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 4 client.15789 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.892+0000 7f4473bf9740 1 -- 172.21.15.110:0/3014004506 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fea1e011f0 con 0x55fea1ea6220 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.893+0000 7f446a59c640 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55fea1ea6220 0x55fea1ea8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.893+0000 7f4451ffb640 10 client.15789 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.893+0000 7f446a59c640 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55fea1ea6220 0x55fea1ea8670 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bff90db5f009052 server_cookie=209a75ec76fe7ef5 in_seq=0 out_seq=0 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4451ffb640 1 -- 172.21.15.110:0/3014004506 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fea1e011f0 con 0x55fea1ea6220 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4451ffb640 10 client.15789 handle_command_reply: tid=0 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 client.15789 shutdown 2024-04-25T00:08:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 -- 172.21.15.110:0/3014004506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e90840 msgr2=0x55fea1e9a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4460009fd0 tx=0x7f446009ba50 comp rx=0 tx=0).stop 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 -- 172.21.15.110:0/3014004506 shutdown_connections 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55fea1ea6220 0x55fea1ea8670 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e959b0 0x55fea1ea1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4451ffb640 0 client.15789 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 --2- 172.21.15.110:0/3014004506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e8fe50 0x55fea1e9a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 -- 172.21.15.110:0/3014004506 >> 172.21.15.110:0/3014004506 conn(0x55fea1e8b4c0 msgr2=0x55fea1e8a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 -- 172.21.15.110:0/3014004506 shutdown_connections 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 1 -- 172.21.15.110:0/3014004506 wait complete. 2024-04-25T00:08:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.894+0000 7f4473bf9740 20 client.15789 trim_cache size 0 max 16384 2024-04-25T00:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.908+0000 7f4473bf9740 1 Processor -- start 2024-04-25T00:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.908+0000 7f4473bf9740 1 -- start start 2024-04-25T00:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.908+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e8fe50 0x55fea1e9a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.908+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.908+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e959b0 0x55fea1e8e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.908+0000 7f4473bf9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1ab3f0 con 0x55fea1e959b0 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.908+0000 7f4473bf9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1a8cc0 con 0x55fea1e90840 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.908+0000 7f4473bf9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1a6b10 con 0x55fea1e8fe50 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e959b0 0x55fea1e8e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e959b0 0x55fea1e8e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51462/0 (socket says 172.21.15.110:51462) 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 -- 172.21.15.110:0/2737907329 learned_addr learned my addr 172.21.15.110:0/2737907329 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446a59c640 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e8fe50 0x55fea1e9a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f4469d9b640 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 -- 172.21.15.110:0/2737907329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e8fe50 msgr2=0x55fea1e9a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e8fe50 0x55fea1e9a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 -- 172.21.15.110:0/2737907329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e90840 msgr2=0x55fea1e9a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 -- 172.21.15.110:0/2737907329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea1e8b060 con 0x55fea1e959b0 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f446ad9d640 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e959b0 0x55fea1e8e470 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f445c005880 tx=0x7f445c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a582d75e4c0a1222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f4453fff640 1 -- 172.21.15.110:0/2737907329 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c005220 con 0x55fea1e959b0 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f4453fff640 1 -- 172.21.15.110:0/2737907329 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f445c004c50 con 0x55fea1e959b0 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.909+0000 7f4453fff640 1 -- 172.21.15.110:0/2737907329 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c01f900 con 0x55fea1e959b0 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- 172.21.15.110:0/2737907329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e959b0 msgr2=0x55fea1e8e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e959b0 0x55fea1e8e470 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f445c005880 tx=0x7f445c01d000 comp rx=0 tx=0).stop 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- 172.21.15.110:0/2737907329 shutdown_connections 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e959b0 0x55fea1e8e470 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e90840 0x55fea1e9a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 --2- 172.21.15.110:0/2737907329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e8fe50 0x55fea1e9a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- 172.21.15.110:0/2737907329 >> 172.21.15.110:0/2737907329 conn(0x55fea1e8b4c0 msgr2=0x55fea1e8b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- 172.21.15.110:0/2737907329 shutdown_connections 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- 172.21.15.110:0/2737907329 wait complete. 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- start start 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 Processor -- start 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1f08320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 0x55fea1f08860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 0x55fea1f0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1a7180 con 0x55fea1e8fe50 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1a85f0 con 0x55fea1e959b0 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1a7850 con 0x55fea1e90840 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4473bf9740 10 client.0 _ll_register_callbacks cb 0x7f44600092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f446a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1f08320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f446a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1f08320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51476/0 (socket says 172.21.15.110:51476) 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f446a59c640 1 -- 172.21.15.110:0/338291592 learned_addr learned my addr 172.21.15.110:0/338291592 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f4469d9b640 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 0x55fea1f08860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.910+0000 7f446a59c640 1 -- 172.21.15.110:0/338291592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 msgr2=0x55fea1f08860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f446a59c640 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 0x55fea1f08860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f446a59c640 1 -- 172.21.15.110:0/338291592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 msgr2=0x55fea1f0f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f446a59c640 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 0x55fea1f0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f446a59c640 1 -- 172.21.15.110:0/338291592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea1f0fe90 con 0x55fea1e8fe50 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f446a59c640 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1f08320 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4464005cc0 tx=0x7f4464009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b543467bd2a24d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4451ffb640 1 -- 172.21.15.110:0/338291592 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464057070 con 0x55fea1e8fe50 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4451ffb640 1 -- 172.21.15.110:0/338291592 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4464003970 con 0x55fea1e8fe50 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 1 -- 172.21.15.110:0/338291592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445c004c50 con 0x55fea1e8fe50 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4451ffb640 1 -- 172.21.15.110:0/338291592 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4464063050 con 0x55fea1e8fe50 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 10 client.15795 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 1 -- 172.21.15.110:0/338291592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fea1f111a0 con 0x55fea1e8fe50 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4451ffb640 1 -- 172.21.15.110:0/338291592 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f445c004c50 con 0x55fea1e8fe50 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4451ffb640 1 -- 172.21.15.110:0/338291592 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f44640305c0 con 0x55fea1e8fe50 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 10 client.15795 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 20 client.15795 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55fea1f11730 0x55fea1f13b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 4 client.15795 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.911+0000 7f4473bf9740 1 -- 172.21.15.110:0/338291592 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fea1dfb540 con 0x55fea1f11730 2024-04-25T00:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.912+0000 7f4469d9b640 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55fea1f11730 0x55fea1f13b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.912+0000 7f4469d9b640 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55fea1f11730 0x55fea1f13b50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a98dc12e8cdb36d server_cookie=6f36d32f33ebd2e8 in_seq=0 out_seq=0 2024-04-25T00:08:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.912+0000 7f4451ffb640 10 client.15795 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4451ffb640 1 -- 172.21.15.110:0/338291592 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fea1dfb540 con 0x55fea1f11730 2024-04-25T00:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4451ffb640 10 client.15795 handle_command_reply: tid=0 2024-04-25T00:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 client.15795 shutdown 2024-04-25T00:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 -- 172.21.15.110:0/338291592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 msgr2=0x55fea1f08320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1f08320 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4464005cc0 tx=0x7f4464009c20 comp rx=0 tx=0).stop 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 -- 172.21.15.110:0/338291592 shutdown_connections 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55fea1f11730 0x55fea1f13b50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fea1e959b0 0x55fea1f0f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4451ffb640 0 client.15795 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fea1e90840 0x55fea1f08860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 --2- 172.21.15.110:0/338291592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fea1e8fe50 0x55fea1f08320 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 -- 172.21.15.110:0/338291592 >> 172.21.15.110:0/338291592 conn(0x55fea1e8b4c0 msgr2=0x55fea1e9a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.925+0000 7f4473bf9740 1 -- 172.21.15.110:0/338291592 shutdown_connections 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.926+0000 7f4473bf9740 1 -- 172.21.15.110:0/338291592 wait complete. 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.926+0000 7f4473bf9740 20 client.15795 trim_cache size 0 max 16384 2024-04-25T00:08:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:08:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:08:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:08:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:08:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 -- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44540a2c30 msgr2=0x7f44540a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44540a2c30 0x7f44540a50f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b590 tx=0x7f445c0127c0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 -- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c100f80 msgr2=0x7f446c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c100f80 0x7f446c19c720 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f446000b450 tx=0x7f446000b910 comp rx=0 tx=0).stop 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 -- 172.21.15.110:0/724980296 shutdown_connections 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f44540a2c30 0x7f44540a50f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c101940 0x7f446c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f446c100f80 0x7f446c19c720 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 --2- 172.21.15.110:0/724980296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c0ffe30 0x7f446c1097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 -- 172.21.15.110:0/724980296 >> 172.21.15.110:0/724980296 conn(0x7f446c0fb3a0 msgr2=0x7f446c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 -- 172.21.15.110:0/724980296 shutdown_connections 2024-04-25T00:08:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:42.927+0000 7f4470e12640 1 -- 172.21.15.110:0/724980296 wait complete. 2024-04-25T00:08:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:43 smithi188 ceph-mon[33285]: pgmap v551: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-04-25T00:08:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:43 smithi110 ceph-mon[29187]: pgmap v551: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-04-25T00:08:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:43 smithi173 ceph-mon[32402]: pgmap v551: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-04-25T00:08:46.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:45 smithi188 ceph-mon[33285]: pgmap v552: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 993 op/s 2024-04-25T00:08:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:45 smithi110 ceph-mon[29187]: pgmap v552: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 993 op/s 2024-04-25T00:08:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:45 smithi173 ceph-mon[32402]: pgmap v552: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 993 op/s 2024-04-25T00:08:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 340.0 s, 728.2 tps, lat 136.877 ms stddev 85.976 2024-04-25T00:08:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:47 smithi188 ceph-mon[33285]: pgmap v553: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-04-25T00:08:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:47 smithi110 ceph-mon[29187]: pgmap v553: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-04-25T00:08:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:47 smithi173 ceph-mon[32402]: pgmap v553: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-04-25T00:08:50.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:49 smithi188 ceph-mon[33285]: pgmap v554: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-04-25T00:08:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:49 smithi110 ceph-mon[29187]: pgmap v554: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-04-25T00:08:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:49 smithi173 ceph-mon[32402]: pgmap v554: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-04-25T00:08:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 345.0 s, 811.2 tps, lat 123.754 ms stddev 11.762 2024-04-25T00:08:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:51 smithi188 ceph-mon[33285]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 956 op/s 2024-04-25T00:08:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:51 smithi110 ceph-mon[29187]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 956 op/s 2024-04-25T00:08:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:51 smithi173 ceph-mon[32402]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 956 op/s 2024-04-25T00:08:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:53 smithi188 ceph-mon[33285]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 986 op/s 2024-04-25T00:08:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:53 smithi110 ceph-mon[29187]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 986 op/s 2024-04-25T00:08:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:53 smithi173 ceph-mon[32402]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 986 op/s 2024-04-25T00:08:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:55 smithi188 ceph-mon[33285]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-04-25T00:08:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:55 smithi110 ceph-mon[29187]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-04-25T00:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 -- 172.21.15.110:0/1609293380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00d0104c10 msgr2=0x7f00d0107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 --2- 172.21.15.110:0/1609293380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00d0104c10 0x7f00d0107000 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009900 tx=0x7f00bc02a410 comp rx=0 tx=0).stop 2024-04-25T00:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 -- 172.21.15.110:0/1609293380 shutdown_connections 2024-04-25T00:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 --2- 172.21.15.110:0/1609293380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d0107540 0x7f00d0109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 --2- 172.21.15.110:0/1609293380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00d0104c10 0x7f00d0107000 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 --2- 172.21.15.110:0/1609293380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d01022e0 0x7f00d01046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 -- 172.21.15.110:0/1609293380 >> 172.21.15.110:0/1609293380 conn(0x7f00d00fbc10 msgr2=0x7f00d00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 -- 172.21.15.110:0/1609293380 shutdown_connections 2024-04-25T00:08:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.058+0000 7f00d71c1640 1 -- 172.21.15.110:0/1609293380 wait complete. 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.059+0000 7f00d71c1640 1 Processor -- start 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.059+0000 7f00d71c1640 1 -- start start 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d71c1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d01022e0 0x7f00d0197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d71c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d0104c10 0x7f00d0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d4f5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d01022e0 0x7f00d0197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d4f5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d01022e0 0x7f00d0197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60974/0 (socket says 172.21.15.110:60974) 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d4f5d640 1 -- 172.21.15.110:0/2985052530 learned_addr learned my addr 172.21.15.110:0/2985052530 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d71c1640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00d0107540 0x7f00d019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00d010d330 con 0x7f00d01022e0 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f00d010d1b0 con 0x7f00d0107540 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00d010d4b0 con 0x7f00d0104c10 2024-04-25T00:08:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.060+0000 7f00cffff640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d0104c10 0x7f00d0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00d575e640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00d0107540 0x7f00d019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00d4f5d640 1 -- 172.21.15.110:0/2985052530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d0104c10 msgr2=0x7f00d0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00d4f5d640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d0104c10 0x7f00d0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00d4f5d640 1 -- 172.21.15.110:0/2985052530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00d0107540 msgr2=0x7f00d019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00d4f5d640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00d0107540 0x7f00d019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00d4f5d640 1 -- 172.21.15.110:0/2985052530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d01000d0 con 0x7f00d01022e0 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00cffff640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d0104c10 0x7f00d0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00d4f5d640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d01022e0 0x7f00d0197980 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f00c400e9f0 tx=0x7f00c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00cdffb640 1 -- 172.21.15.110:0/2985052530 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c400cd80 con 0x7f00d01022e0 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d01003c0 con 0x7f00d01022e0 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00cdffb640 1 -- 172.21.15.110:0/2985052530 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c40108e0 con 0x7f00d01022e0 2024-04-25T00:08:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.061+0000 7f00cdffb640 1 -- 172.21.15.110:0/2985052530 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4018920 con 0x7f00d01022e0 2024-04-25T00:08:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.062+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d01008e0 con 0x7f00d01022e0 2024-04-25T00:08:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d9fa9740 1 Processor -- start 2024-04-25T00:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d9fa9740 1 -- start start 2024-04-25T00:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x556438081a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x556438086b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 0x556438089650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556437fdccb0 con 0x556438082050 2024-04-25T00:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556437f855b0 con 0x5564380871c0 2024-04-25T00:08:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556437fd8c70 con 0x556438081660 2024-04-25T00:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00cdffb640 1 -- 172.21.15.110:0/2985052530 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f00c4010a90 con 0x7f00d01022e0 2024-04-25T00:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d575e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x556438081a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d575e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x556438081a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38278/0 (socket says 172.21.15.110:38278) 2024-04-25T00:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.065+0000 7f00d575e640 1 -- 172.21.15.110:0/3481786407 learned_addr learned my addr 172.21.15.110:0/3481786407 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00cffff640 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 0x556438089650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d4f5d640 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x556438086b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d575e640 1 -- 172.21.15.110:0/3481786407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 msgr2=0x556438089650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d575e640 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 0x556438089650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d575e640 1 -- 172.21.15.110:0/3481786407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 msgr2=0x556438086b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d575e640 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x556438086b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d575e640 1 -- 172.21.15.110:0/3481786407 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556437cf1210 con 0x556438081660 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d4f5d640 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x556438086b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00cdffb640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f00c00a2c30 0x7f00c00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d575e640 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x556438081a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f00c8009fc0 tx=0x7f00c8010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f5a5ee6ced5765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00d4f5d640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f00c00a2c30 0x7f00c00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00baffd640 1 -- 172.21.15.110:0/3481786407 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8003b70 con 0x556438081660 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00baffd640 1 -- 172.21.15.110:0/3481786407 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c8003d10 con 0x556438081660 2024-04-25T00:08:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00baffd640 1 -- 172.21.15.110:0/3481786407 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8011a50 con 0x556438081660 2024-04-25T00:08:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.066+0000 7f00cdffb640 1 -- 172.21.15.110:0/2985052530 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f00c40cf3f0 con 0x7f00d01022e0 2024-04-25T00:08:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.067+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3481786407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 msgr2=0x556438081a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.067+0000 7f00d4f5d640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f00c00a2c30 0x7f00c00a50f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f00c409b920 tx=0x7f00c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.067+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x556438081a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f00c8009fc0 tx=0x7f00c8010860 comp rx=0 tx=0).stop 2024-04-25T00:08:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.067+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3481786407 shutdown_connections 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.068+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 0x556438089650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.068+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x556438086b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.068+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3481786407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x556438081a80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.068+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3481786407 >> 172.21.15.110:0/3481786407 conn(0x5564380744c0 msgr2=0x5564380748c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.068+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3481786407 shutdown_connections 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.068+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3481786407 wait complete. 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.068+0000 7f00d9fa9740 1 -- start start 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.068+0000 7f00d9fa9740 1 Processor -- start 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x55643808ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x55643808bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 0x556438093010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55643808a170 con 0x556438082050 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556437f98b50 con 0x5564380871c0 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55643808a2f0 con 0x556438081660 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d9fa9740 10 client.0 _ll_register_callbacks cb 0x556437fd3ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d575e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x55643808ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00cffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x55643808bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00d575e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 0x556438093010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00cffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x55643808bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60998/0 (socket says 172.21.15.110:60998) 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.069+0000 7f00cffff640 1 -- 172.21.15.110:0/3835055633 learned_addr learned my addr 172.21.15.110:0/3835055633 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.070+0000 7f00d575e640 1 -- 172.21.15.110:0/3835055633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 msgr2=0x556438093010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.070+0000 7f00d575e640 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 0x556438093010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.070+0000 7f00d575e640 1 -- 172.21.15.110:0/3835055633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 msgr2=0x55643808bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.070+0000 7f00d575e640 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x55643808bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.070+0000 7f00d575e640 1 -- 172.21.15.110:0/3835055633 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556437dbd620 con 0x556438081660 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.070+0000 7f00d575e640 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x55643808ba90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f00c8003ce0 tx=0x7f00c8010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947769a00593cc15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.070+0000 7f00affff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.070+0000 7f00affff640 1 -- 172.21.15.110:0/3835055633 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8011c80 con 0x556438081660 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00affff640 1 -- 172.21.15.110:0/3835055633 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f00c8003ec0 con 0x556438081660 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3835055633 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c8003d10 con 0x556438081660 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00affff640 1 -- 172.21.15.110:0/3835055633 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f00c8011e20 con 0x556438081660 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 10 client.25363 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3835055633 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564380962f0 con 0x556438081660 2024-04-25T00:08:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00affff640 1 -- 172.21.15.110:0/3835055633 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c8003d10 con 0x556438081660 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00affff640 1 -- 172.21.15.110:0/3835055633 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f00c8029040 con 0x556438081660 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 10 client.25363 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 20 client.25363 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5564380977a0 0x556438099b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 4 client.25363 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.071+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3835055633 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556437fea1f0 con 0x5564380977a0 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.072+0000 7f00cffff640 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5564380977a0 0x556438099b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.072+0000 7f00affff640 10 client.25363 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.072+0000 7f00cffff640 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5564380977a0 0x556438099b90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9614f1281979e1f server_cookie=77c97be7baa74c2e in_seq=0 out_seq=0 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.073+0000 7f00affff640 1 -- 172.21.15.110:0/3835055633 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556437fea1f0 con 0x5564380977a0 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.073+0000 7f00affff640 10 client.25363 handle_command_reply: tid=0 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.073+0000 7f00d9fa9740 1 client.25363 shutdown 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.073+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3835055633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 msgr2=0x55643808ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.073+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x55643808ba90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f00c8003ce0 tx=0x7f00c8010fd0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.073+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3835055633 shutdown_connections 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.073+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5564380977a0 0x556438099b90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.074+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564380871c0 0x556438093010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.074+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438082050 0x55643808bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.074+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/3835055633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438081660 0x55643808ba90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.074+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3835055633 >> 172.21.15.110:0/3835055633 conn(0x5564380744c0 msgr2=0x5564380735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.074+0000 7f00affff640 0 client.25363 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.074+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3835055633 shutdown_connections 2024-04-25T00:08:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.074+0000 7f00d9fa9740 1 -- 172.21.15.110:0/3835055633 wait complete. 2024-04-25T00:08:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.074+0000 7f00d9fa9740 20 client.25363 trim_cache size 0 max 16384 2024-04-25T00:08:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.097+0000 7f00d9fa9740 1 Processor -- start 2024-04-25T00:08:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.098+0000 7f00d9fa9740 1 -- start start 2024-04-25T00:08:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.098+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438081660 0x556438099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.098+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438082050 0x556438099ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.098+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564380871c0 0x55643808b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.098+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a91b0 con 0x556438081660 2024-04-25T00:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.098+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a6a80 con 0x556438082050 2024-04-25T00:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.098+0000 7f00d575e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564380871c0 0x55643808b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564380871c0 0x55643808b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38288/0 (socket says 172.21.15.110:38288) 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00cffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438082050 0x556438099ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 -- 172.21.15.110:0/2451520601 learned_addr learned my addr 172.21.15.110:0/2451520601 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d9fa9740 1 -- 172.21.15.110:0/2451520601 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a48d0 con 0x5564380871c0 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438081660 0x556438099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 -- 172.21.15.110:0/2451520601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438082050 msgr2=0x556438099ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438082050 0x556438099ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 -- 172.21.15.110:0/2451520601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438081660 msgr2=0x556438099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438081660 0x556438099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 -- 172.21.15.110:0/2451520601 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556438086730 con 0x5564380871c0 2024-04-25T00:08:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00cffff640 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438082050 0x556438099ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.099+0000 7f00d575e640 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564380871c0 0x55643808b6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f00c8004e10 tx=0x7f00c8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57f9c2523cdcf3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.100+0000 7f00ba7fc640 1 -- 172.21.15.110:0/2451520601 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8011c80 con 0x5564380871c0 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.100+0000 7f00ba7fc640 1 -- 172.21.15.110:0/2451520601 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00c80099c0 con 0x5564380871c0 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.100+0000 7f00ba7fc640 1 -- 172.21.15.110:0/2451520601 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8003d10 con 0x5564380871c0 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 -- 172.21.15.110:0/2451520601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564380871c0 msgr2=0x55643808b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564380871c0 0x55643808b6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f00c8004e10 tx=0x7f00c8005fd0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 -- 172.21.15.110:0/2451520601 shutdown_connections 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564380871c0 0x55643808b6a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438082050 0x556438099ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/2451520601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556438081660 0x556438099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 -- 172.21.15.110:0/2451520601 >> 172.21.15.110:0/2451520601 conn(0x5564380744c0 msgr2=0x5564380748a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 -- 172.21.15.110:0/2451520601 shutdown_connections 2024-04-25T00:08:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.101+0000 7f00d9fa9740 1 -- 172.21.15.110:0/2451520601 wait complete. 2024-04-25T00:08:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 1 -- start start 2024-04-25T00:08:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 1 Processor -- start 2024-04-25T00:08:56.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:55 smithi173 ceph-mon[32402]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-04-25T00:08:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438081660 0x55643808f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438082050 0x55643808fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564380871c0 0x556438090c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a4f40 con 0x5564380871c0 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a63b0 con 0x556438081660 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a5610 con 0x556438082050 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.102+0000 7f00d9fa9740 10 client.0 _ll_register_callbacks cb 0x7f00c8009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00d575e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438081660 0x55643808f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00cffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438082050 0x55643808fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00d575e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438081660 0x55643808f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39378/0 (socket says 172.21.15.110:39378) 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00d575e640 1 -- 172.21.15.110:0/638080188 learned_addr learned my addr 172.21.15.110:0/638080188 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00d575e640 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564380871c0 0x556438090c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00cffff640 1 -- 172.21.15.110:0/638080188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438081660 msgr2=0x55643808f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00cffff640 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438081660 0x55643808f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00cffff640 1 -- 172.21.15.110:0/638080188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564380871c0 msgr2=0x556438090c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00cffff640 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564380871c0 0x556438090c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00cffff640 1 -- 172.21.15.110:0/638080188 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556438091210 con 0x556438082050 2024-04-25T00:08:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00cffff640 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438082050 0x55643808fca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0314e0 tx=0x7f00bc003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21dbbebb3c1f7915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.103+0000 7f00affff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.104+0000 7f00affff640 1 -- 172.21.15.110:0/638080188 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc04a030 con 0x556438082050 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.104+0000 7f00affff640 1 -- 172.21.15.110:0/638080188 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f00bc003910 con 0x556438082050 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.104+0000 7f00d9fa9740 1 -- 172.21.15.110:0/638080188 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c80099c0 con 0x556438082050 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.104+0000 7f00affff640 1 -- 172.21.15.110:0/638080188 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f00bc035040 con 0x556438082050 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.104+0000 7f00d9fa9740 10 client.25369 fetch_fsmap learned FSMap version 32 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.104+0000 7f00d9fa9740 1 -- 172.21.15.110:0/638080188 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556438092590 con 0x556438082050 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.104+0000 7f00affff640 1 -- 172.21.15.110:0/638080188 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00bc0462c0 con 0x556438082050 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00affff640 1 -- 172.21.15.110:0/638080188 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f00bc031650 con 0x556438082050 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00d9fa9740 10 client.25369 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00d9fa9740 10 client.25369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00d9fa9740 20 client.25369 populate_metadata read hostname 'smithi110' 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5564380fe0d0 0x5564381004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00d9fa9740 4 client.25369 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:08:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00d9fa9740 1 -- 172.21.15.110:0/638080188 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f00c80099c0 con 0x5564380fe0d0 2024-04-25T00:08:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00d575e640 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5564380fe0d0 0x5564381004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00affff640 10 client.25369 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.105+0000 7f00d575e640 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5564380fe0d0 0x5564381004f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4eee8bdce2f82f server_cookie=5d999f442ae97de9 in_seq=0 out_seq=0 2024-04-25T00:08:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.124+0000 7f00affff640 1 -- 172.21.15.110:0/638080188 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f00c80099c0 con 0x5564380fe0d0 2024-04-25T00:08:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.124+0000 7f00affff640 10 client.25369 handle_command_reply: tid=0 2024-04-25T00:08:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.124+0000 7f00d9fa9740 1 client.25369 shutdown 2024-04-25T00:08:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 -- 172.21.15.110:0/638080188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438082050 msgr2=0x55643808fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438082050 0x55643808fca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0314e0 tx=0x7f00bc003820 comp rx=0 tx=0).stop 2024-04-25T00:08:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 -- 172.21.15.110:0/638080188 shutdown_connections 2024-04-25T00:08:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5564380fe0d0 0x5564381004f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564380871c0 0x556438090c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556438082050 0x55643808fca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00affff640 0 client.25369 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 --2- 172.21.15.110:0/638080188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556438081660 0x55643808f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 -- 172.21.15.110:0/638080188 >> 172.21.15.110:0/638080188 conn(0x5564380744c0 msgr2=0x5564380834b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 -- 172.21.15.110:0/638080188 shutdown_connections 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 1 -- 172.21.15.110:0/638080188 wait complete. 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.125+0000 7f00d9fa9740 20 client.25369 trim_cache size 0 max 16384 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:08:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:08:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:08:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:08:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f00c00a2c30 msgr2=0x7f00c00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f00c00a2c30 0x7f00c00a50f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f00c409b920 tx=0x7f00c4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d01022e0 msgr2=0x7f00d0197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d01022e0 0x7f00d0197980 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f00c400e9f0 tx=0x7f00c400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 shutdown_connections 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f00c00a2c30 0x7f00c00a50f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00d0107540 0x7f00d019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d0104c10 0x7f00d0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 --2- 172.21.15.110:0/2985052530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f00d01022e0 0x7f00d0197980 secure :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f00c400e9f0 tx=0x7f00c400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 >> 172.21.15.110:0/2985052530 conn(0x7f00d00fbc10 msgr2=0x7f00d0102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.126+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 shutdown_connections 2024-04-25T00:08:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:08:56.127+0000 7f00d71c1640 1 -- 172.21.15.110:0/2985052530 wait complete. 2024-04-25T00:08:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 350.0 s, 774.0 tps, lat 129.208 ms stddev 80.195 2024-04-25T00:08:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:57 smithi188 ceph-mon[33285]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.9 MiB/s wr, 853 op/s 2024-04-25T00:08:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:57 smithi110 ceph-mon[29187]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.9 MiB/s wr, 853 op/s 2024-04-25T00:08:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:57 smithi173 ceph-mon[32402]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.9 MiB/s wr, 853 op/s 2024-04-25T00:09:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:08:59 smithi188 ceph-mon[33285]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-04-25T00:09:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:08:59 smithi110 ceph-mon[29187]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-04-25T00:09:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:08:59 smithi173 ceph-mon[32402]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-04-25T00:09:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 355.0 s, 819.0 tps, lat 121.864 ms stddev 21.409 2024-04-25T00:09:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:01 smithi188 ceph-mon[33285]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 876 op/s 2024-04-25T00:09:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:01 smithi110 ceph-mon[29187]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 876 op/s 2024-04-25T00:09:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:01 smithi173 ceph-mon[32402]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 876 op/s 2024-04-25T00:09:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:03 smithi188 ceph-mon[33285]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 991 op/s 2024-04-25T00:09:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:03 smithi110 ceph-mon[29187]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 991 op/s 2024-04-25T00:09:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:03 smithi173 ceph-mon[32402]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 991 op/s 2024-04-25T00:09:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:05 smithi188 ceph-mon[33285]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-04-25T00:09:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:05 smithi110 ceph-mon[29187]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-04-25T00:09:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:05 smithi173 ceph-mon[32402]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-04-25T00:09:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 360.0 s, 881.6 tps, lat 113.202 ms stddev 15.882 2024-04-25T00:09:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:07 smithi188 ceph-mon[33285]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 976 op/s 2024-04-25T00:09:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:07 smithi110 ceph-mon[29187]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 976 op/s 2024-04-25T00:09:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:07 smithi173 ceph-mon[32402]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 976 op/s 2024-04-25T00:09:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 -- 172.21.15.110:0/937047680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8640ff640 msgr2=0x7fe8640ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 --2- 172.21.15.110:0/937047680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8640ff640 0x7fe8640ffa60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe858009860 tx=0x7fe85802a340 comp rx=0 tx=0).stop 2024-04-25T00:09:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 -- 172.21.15.110:0/937047680 shutdown_connections 2024-04-25T00:09:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 --2- 172.21.15.110:0/937047680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe864101110 0x7fe8641059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 --2- 172.21.15.110:0/937047680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe864100770 0x7fe864100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 --2- 172.21.15.110:0/937047680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8640ff640 0x7fe8640ffa60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 -- 172.21.15.110:0/937047680 >> 172.21.15.110:0/937047680 conn(0x7fe8640faf60 msgr2=0x7fe8640fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 -- 172.21.15.110:0/937047680 shutdown_connections 2024-04-25T00:09:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.266+0000 7fe86a51b640 1 -- 172.21.15.110:0/937047680 wait complete. 2024-04-25T00:09:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.267+0000 7fe86a51b640 1 Processor -- start 2024-04-25T00:09:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.267+0000 7fe86a51b640 1 -- start start 2024-04-25T00:09:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.267+0000 7fe86a51b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8640ff640 0x7fe86419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.267+0000 7fe86a51b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe864100770 0x7fe86419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.267+0000 7fe86a51b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe864101110 0x7fe8641a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.267+0000 7fe86a51b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe864113890 con 0x7fe8640ff640 2024-04-25T00:09:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.267+0000 7fe86a51b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe864113710 con 0x7fe864100770 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.267+0000 7fe86a51b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe864113a10 con 0x7fe864101110 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe863fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8640ff640 0x7fe86419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe863fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8640ff640 0x7fe86419df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52962/0 (socket says 172.21.15.110:52962) 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe863fff640 1 -- 172.21.15.110:0/567612432 learned_addr learned my addr 172.21.15.110:0/567612432 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe868ab8640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe864101110 0x7fe8641a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe8637fe640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe864100770 0x7fe86419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe868ab8640 1 -- 172.21.15.110:0/567612432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe864100770 msgr2=0x7fe86419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe868ab8640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe864100770 0x7fe86419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe868ab8640 1 -- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8640ff640 msgr2=0x7fe86419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe868ab8640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8640ff640 0x7fe86419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe868ab8640 1 -- 172.21.15.110:0/567612432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8641a5b20 con 0x7fe864101110 2024-04-25T00:09:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.268+0000 7fe868ab8640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe864101110 0x7fe8641a54b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009fd0 tx=0x7fe85c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.269+0000 7fe8617fa640 1 -- 172.21.15.110:0/567612432 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00ce20 con 0x7fe864101110 2024-04-25T00:09:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.269+0000 7fe86a51b640 1 -- 172.21.15.110:0/567612432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8641a5e10 con 0x7fe864101110 2024-04-25T00:09:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.269+0000 7fe86a51b640 1 -- 172.21.15.110:0/567612432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8641a62d0 con 0x7fe864101110 2024-04-25T00:09:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.269+0000 7fe8617fa640 1 -- 172.21.15.110:0/567612432 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85c0108e0 con 0x7fe864101110 2024-04-25T00:09:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.269+0000 7fe8617fa640 1 -- 172.21.15.110:0/567612432 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00fa60 con 0x7fe864101110 2024-04-25T00:09:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.273+0000 7fe86d22b740 1 Processor -- start 2024-04-25T00:09:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.274+0000 7fe8617fa640 1 -- 172.21.15.110:0/567612432 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe85c00fc00 con 0x7fe864101110 2024-04-25T00:09:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.274+0000 7fe86d22b740 1 -- start start 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe8617fa640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe84c0a2cb0 0x7fe84c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91be50 0x564d9b91c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b921340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b923e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe86d22b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564d9b87fcb0 con 0x564d9b9219b0 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe86d22b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564d9b8285b0 con 0x564d9b91be50 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe86d22b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d9b87bc70 con 0x564d9b91c840 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe8617fa640 1 -- 172.21.15.110:0/567612432 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe85c0cf220 con 0x7fe864101110 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe868ab8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b923e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe863fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91be50 0x564d9b91c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe8637fe640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe84c0a2cb0 0x7fe84c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe868ab8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b923e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52978/0 (socket says 172.21.15.110:52978) 2024-04-25T00:09:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe868ab8640 1 -- 172.21.15.110:0/486532860 learned_addr learned my addr 172.21.15.110:0/486532860 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.275+0000 7fe8637fe640 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b921340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe868ab8640 1 -- 172.21.15.110:0/486532860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 msgr2=0x564d9b921340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe868ab8640 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b921340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe868ab8640 1 -- 172.21.15.110:0/486532860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91be50 msgr2=0x564d9b91c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe868ab8640 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91be50 0x564d9b91c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe868ab8640 1 -- 172.21.15.110:0/486532860 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9b594210 con 0x564d9b9219b0 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe8637fe640 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b921340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe863fff640 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91be50 0x564d9b91c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe868ab8640 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b923e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe85c098480 tx=0x7fe85c09c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ba7d8bce28ab5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe8637fe640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe84c0a2cb0 0x7fe84c0a5170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x564d9b91cc40 tx=0x7fe854009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe84a7fc640 1 -- 172.21.15.110:0/486532860 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0a4720 con 0x564d9b9219b0 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.276+0000 7fe84a7fc640 1 -- 172.21.15.110:0/486532860 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85c099b50 con 0x564d9b9219b0 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.277+0000 7fe84a7fc640 1 -- 172.21.15.110:0/486532860 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c087e30 con 0x564d9b9219b0 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.277+0000 7fe86d22b740 1 -- 172.21.15.110:0/486532860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 msgr2=0x564d9b923e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.277+0000 7fe86d22b740 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b923e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe85c098480 tx=0x7fe85c09c1b0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.278+0000 7fe86d22b740 1 -- 172.21.15.110:0/486532860 shutdown_connections 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.278+0000 7fe86d22b740 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b923e40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.278+0000 7fe86d22b740 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b921340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.278+0000 7fe86d22b740 1 --2- 172.21.15.110:0/486532860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91be50 0x564d9b91c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.278+0000 7fe86d22b740 1 -- 172.21.15.110:0/486532860 >> 172.21.15.110:0/486532860 conn(0x564d9b9174c0 msgr2=0x564d9b9178c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.278+0000 7fe86d22b740 1 -- 172.21.15.110:0/486532860 shutdown_connections 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.278+0000 7fe86d22b740 1 -- 172.21.15.110:0/486532860 wait complete. 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.278+0000 7fe86d22b740 1 -- start start 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.279+0000 7fe86d22b740 1 Processor -- start 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.279+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b91be50 0x564d9b926120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.279+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91c840 0x564d9b926660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.279+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b9219b0 0x564d9b92d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.279+0000 7fe86d22b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564d9b924960 con 0x564d9b91be50 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.279+0000 7fe86d22b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564d9b83bb50 con 0x564d9b91c840 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.279+0000 7fe863fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b91be50 0x564d9b926120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.279+0000 7fe86d22b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d9b924ae0 con 0x564d9b9219b0 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe863fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b91be50 0x564d9b926120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52986/0 (socket says 172.21.15.110:52986) 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe863fff640 1 -- 172.21.15.110:0/1847472644 learned_addr learned my addr 172.21.15.110:0/1847472644 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe86d22b740 10 client.0 _ll_register_callbacks cb 0x564d9b876ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe868ab8640 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b9219b0 0x564d9b92d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe8637fe640 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91c840 0x564d9b926660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe863fff640 1 -- 172.21.15.110:0/1847472644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b9219b0 msgr2=0x564d9b92d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe863fff640 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b9219b0 0x564d9b92d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe863fff640 1 -- 172.21.15.110:0/1847472644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91c840 msgr2=0x564d9b926660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe863fff640 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91c840 0x564d9b926660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe863fff640 1 -- 172.21.15.110:0/1847472644 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9b660620 con 0x564d9b91be50 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe868ab8640 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b9219b0 0x564d9b92d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe863fff640 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b91be50 0x564d9b926120 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe8580057c0 tx=0x7fe858030780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a3a33ed5d6c0f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.280+0000 7fe8437fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.281+0000 7fe8437fe640 1 -- 172.21.15.110:0/1847472644 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858037040 con 0x564d9b91be50 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.281+0000 7fe86d22b740 1 -- 172.21.15.110:0/1847472644 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe85c099b50 con 0x564d9b91be50 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.281+0000 7fe8437fe640 1 -- 172.21.15.110:0/1847472644 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe858004450 con 0x564d9b91be50 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.281+0000 7fe8437fe640 1 -- 172.21.15.110:0/1847472644 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8580035f0 con 0x564d9b91be50 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.281+0000 7fe86d22b740 10 client.15822 fetch_fsmap learned FSMap version 32 2024-04-25T00:09:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.281+0000 7fe86d22b740 1 -- 172.21.15.110:0/1847472644 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d9b930a50 con 0x564d9b91be50 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe8437fe640 1 -- 172.21.15.110:0/1847472644 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85c099b50 con 0x564d9b91be50 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe8437fe640 1 -- 172.21.15.110:0/1847472644 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe8580327c0 con 0x564d9b91be50 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe86d22b740 10 client.15822 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe86d22b740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe86d22b740 20 client.15822 populate_metadata read hostname 'smithi110' 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe86d22b740 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564d9b9321b0 0x564d9b934620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe86d22b740 4 client.15822 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe86d22b740 1 -- 172.21.15.110:0/1847472644 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d9b88d1f0 con 0x564d9b9321b0 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.282+0000 7fe8637fe640 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564d9b9321b0 0x564d9b934620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.283+0000 7fe8437fe640 10 client.15822 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.283+0000 7fe8637fe640 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564d9b9321b0 0x564d9b934620 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e6d4309b4fcbfb server_cookie=205a683d9d7a3ec3 in_seq=0 out_seq=0 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.284+0000 7fe8437fe640 1 -- 172.21.15.110:0/1847472644 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d9b88d1f0 con 0x564d9b9321b0 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.284+0000 7fe8437fe640 10 client.15822 handle_command_reply: tid=0 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.284+0000 7fe86d22b740 1 client.15822 shutdown 2024-04-25T00:09:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.284+0000 7fe86d22b740 1 -- 172.21.15.110:0/1847472644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b91be50 msgr2=0x564d9b926120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.284+0000 7fe86d22b740 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b91be50 0x564d9b926120 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe8580057c0 tx=0x7fe858030780 comp rx=0 tx=0).stop 2024-04-25T00:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.285+0000 7fe86d22b740 1 -- 172.21.15.110:0/1847472644 shutdown_connections 2024-04-25T00:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.285+0000 7fe86d22b740 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564d9b9321b0 0x564d9b934620 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.285+0000 7fe86d22b740 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b9219b0 0x564d9b92d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.285+0000 7fe8437fe640 0 client.15822 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.285+0000 7fe86d22b740 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b91c840 0x564d9b926660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.285+0000 7fe86d22b740 1 --2- 172.21.15.110:0/1847472644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b91be50 0x564d9b926120 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe8580057c0 tx=0x7fe858030780 comp rx=0 tx=0).stop 2024-04-25T00:09:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.285+0000 7fe86d22b740 1 -- 172.21.15.110:0/1847472644 >> 172.21.15.110:0/1847472644 conn(0x564d9b9174c0 msgr2=0x564d9b9165c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.286+0000 7fe86d22b740 1 -- 172.21.15.110:0/1847472644 shutdown_connections 2024-04-25T00:09:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.286+0000 7fe86d22b740 1 -- 172.21.15.110:0/1847472644 wait complete. 2024-04-25T00:09:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.286+0000 7fe86d22b740 20 client.15822 trim_cache size 0 max 16384 2024-04-25T00:09:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe86d22b740 1 Processor -- start 2024-04-25T00:09:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe86d22b740 1 -- start start 2024-04-25T00:09:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b91acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9219b0 0x564d9b92d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9321b0 0x564d9b925770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe86d22b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe850010c70 con 0x564d9b9321b0 2024-04-25T00:09:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe86d22b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe85000e540 con 0x564d9b9219b0 2024-04-25T00:09:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe863fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b91acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe86d22b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe85000c390 con 0x564d9b91c840 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe863fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b91acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:40058/0 (socket says 172.21.15.110:40058) 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe863fff640 1 -- 172.21.15.110:0/2323684875 learned_addr learned my addr 172.21.15.110:0/2323684875 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe8637fe640 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9219b0 0x564d9b92d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe868ab8640 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9321b0 0x564d9b925770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe8637fe640 1 -- 172.21.15.110:0/2323684875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 msgr2=0x564d9b91acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.315+0000 7fe8637fe640 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b91acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.316+0000 7fe8637fe640 1 -- 172.21.15.110:0/2323684875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9321b0 msgr2=0x564d9b925770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.316+0000 7fe8637fe640 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9321b0 0x564d9b925770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.316+0000 7fe8637fe640 1 -- 172.21.15.110:0/2323684875 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9b930870 con 0x564d9b9219b0 2024-04-25T00:09:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.316+0000 7fe8637fe640 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9219b0 0x564d9b92d630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x564d9b91d2b0 tx=0x7fe854004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dea84bdc7b809ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.316+0000 7fe849ffb640 1 -- 172.21.15.110:0/2323684875 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85401a070 con 0x564d9b9219b0 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.316+0000 7fe849ffb640 1 -- 172.21.15.110:0/2323684875 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85400fdb0 con 0x564d9b9219b0 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.316+0000 7fe849ffb640 1 -- 172.21.15.110:0/2323684875 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854020a30 con 0x564d9b9219b0 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 -- 172.21.15.110:0/2323684875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9219b0 msgr2=0x564d9b92d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9219b0 0x564d9b92d630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x564d9b91d2b0 tx=0x7fe854004730 comp rx=0 tx=0).stop 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 -- 172.21.15.110:0/2323684875 shutdown_connections 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9321b0 0x564d9b925770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9219b0 0x564d9b92d630 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 --2- 172.21.15.110:0/2323684875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b91acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 -- 172.21.15.110:0/2323684875 >> 172.21.15.110:0/2323684875 conn(0x564d9b9174c0 msgr2=0x564d9b9178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 -- 172.21.15.110:0/2323684875 shutdown_connections 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.317+0000 7fe86d22b740 1 -- 172.21.15.110:0/2323684875 wait complete. 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.318+0000 7fe86d22b740 1 -- start start 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.318+0000 7fe86d22b740 1 Processor -- start 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.318+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b92aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.318+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b92b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.318+0000 7fe86d22b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9321b0 0x564d9b92c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.318+0000 7fe86d22b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe85000ca00 con 0x564d9b9219b0 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.318+0000 7fe86d22b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe85000de70 con 0x564d9b9321b0 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.318+0000 7fe86d22b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe85000d0d0 con 0x564d9b91c840 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe86d22b740 10 client.0 _ll_register_callbacks cb 0x7fe85c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9321b0 0x564d9b92c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9321b0 0x564d9b92c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:53624/0 (socket says 172.21.15.110:53624) 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 -- 172.21.15.110:0/3455855098 learned_addr learned my addr 172.21.15.110:0/3455855098 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe863fff640 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b92aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe8637fe640 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b92b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 -- 172.21.15.110:0/3455855098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 msgr2=0x564d9b92aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b92aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 -- 172.21.15.110:0/3455855098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 msgr2=0x564d9b92b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b92b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 -- 172.21.15.110:0/3455855098 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9b92c630 con 0x564d9b9321b0 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe868ab8640 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9321b0 0x564d9b92c040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe85c097f10 tx=0x7fe85c0a4720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6251955a2b2db3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe8437fe640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe8437fe640 1 -- 172.21.15.110:0/3455855098 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c09b100 con 0x564d9b9321b0 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe86d22b740 1 -- 172.21.15.110:0/3455855098 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe85400fdb0 con 0x564d9b9321b0 2024-04-25T00:09:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.319+0000 7fe8437fe640 1 -- 172.21.15.110:0/3455855098 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fe85c097800 con 0x564d9b9321b0 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe8437fe640 1 -- 172.21.15.110:0/3455855098 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe85c010700 con 0x564d9b9321b0 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe86d22b740 10 client.25412 fetch_fsmap learned FSMap version 32 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe86d22b740 1 -- 172.21.15.110:0/3455855098 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d9b92d940 con 0x564d9b9321b0 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe8437fe640 1 -- 172.21.15.110:0/3455855098 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85c097a00 con 0x564d9b9321b0 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe8437fe640 1 -- 172.21.15.110:0/3455855098 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe85c01f020 con 0x564d9b9321b0 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe86d22b740 10 client.25412 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe86d22b740 10 client.25412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe86d22b740 20 client.25412 populate_metadata read hostname 'smithi110' 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe86d22b740 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564d9b998960 0x564d9b99ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe86d22b740 4 client.25412 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.320+0000 7fe86d22b740 1 -- 172.21.15.110:0/3455855098 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe85400fdb0 con 0x564d9b998960 2024-04-25T00:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.321+0000 7fe863fff640 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564d9b998960 0x564d9b99ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.322+0000 7fe8437fe640 10 client.25412 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.322+0000 7fe863fff640 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564d9b998960 0x564d9b99ad50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0e56f090a64c6d server_cookie=37cc1619904e2c67 in_seq=0 out_seq=0 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.354+0000 7fe8437fe640 1 -- 172.21.15.110:0/3455855098 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe85400fdb0 con 0x564d9b998960 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.354+0000 7fe8437fe640 10 client.25412 handle_command_reply: tid=0 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.354+0000 7fe86d22b740 1 client.25412 shutdown 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 -- 172.21.15.110:0/3455855098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9321b0 msgr2=0x564d9b92c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9321b0 0x564d9b92c040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe85c097f10 tx=0x7fe85c0a4720 comp rx=0 tx=0).stop 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 -- 172.21.15.110:0/3455855098 shutdown_connections 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564d9b998960 0x564d9b99ad50 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe8437fe640 0 client.25412 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9b9321b0 0x564d9b92c040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d9b9219b0 0x564d9b92b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 --2- 172.21.15.110:0/3455855098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d9b91c840 0x564d9b92aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 -- 172.21.15.110:0/3455855098 >> 172.21.15.110:0/3455855098 conn(0x564d9b9174c0 msgr2=0x564d9b915d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 -- 172.21.15.110:0/3455855098 shutdown_connections 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 1 -- 172.21.15.110:0/3455855098 wait complete. 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.355+0000 7fe86d22b740 20 client.25412 trim_cache size 0 max 16384 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 -- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe84c0a2cb0 msgr2=0x7fe84c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe84c0a2cb0 0x7fe84c0a5170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x564d9b91cc40 tx=0x7fe854009680 comp rx=0 tx=0).stop 2024-04-25T00:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 -- 172.21.15.110:0/567612432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe864101110 msgr2=0x7fe8641a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe864101110 0x7fe8641a54b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009fd0 tx=0x7fe85c00ef10 comp rx=0 tx=0).stop 2024-04-25T00:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 -- 172.21.15.110:0/567612432 shutdown_connections 2024-04-25T00:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fe84c0a2cb0 0x7fe84c0a5170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe864101110 0x7fe8641a54b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe864100770 0x7fe86419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 --2- 172.21.15.110:0/567612432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8640ff640 0x7fe86419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 -- 172.21.15.110:0/567612432 >> 172.21.15.110:0/567612432 conn(0x7fe8640faf60 msgr2=0x7fe8641019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 -- 172.21.15.110:0/567612432 shutdown_connections 2024-04-25T00:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:09.356+0000 7fe86a51b640 1 -- 172.21.15.110:0/567612432 wait complete. 2024-04-25T00:09:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:09 smithi188 ceph-mon[33285]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 13 MiB/s wr, 1.21k op/s 2024-04-25T00:09:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:09 smithi110 ceph-mon[29187]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 13 MiB/s wr, 1.21k op/s 2024-04-25T00:09:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:09 smithi173 ceph-mon[32402]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 13 MiB/s wr, 1.21k op/s 2024-04-25T00:09:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 365.0 s, 835.2 tps, lat 120.065 ms stddev 16.566 2024-04-25T00:09:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:11 smithi188 ceph-mon[33285]: pgmap v565: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.9 MiB/s wr, 930 op/s 2024-04-25T00:09:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:11 smithi110 ceph-mon[29187]: pgmap v565: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.9 MiB/s wr, 930 op/s 2024-04-25T00:09:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:11 smithi173 ceph-mon[32402]: pgmap v565: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 8.9 MiB/s wr, 930 op/s 2024-04-25T00:09:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:13 smithi188 ceph-mon[33285]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-04-25T00:09:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:13 smithi110 ceph-mon[29187]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-04-25T00:09:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:13 smithi173 ceph-mon[32402]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 10 MiB/s wr, 1.09k op/s 2024-04-25T00:09:16.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:15 smithi188 ceph-mon[33285]: pgmap v567: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-04-25T00:09:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:15 smithi110 ceph-mon[29187]: pgmap v567: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-04-25T00:09:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:15 smithi173 ceph-mon[32402]: pgmap v567: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-04-25T00:09:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 370.0 s, 832.8 tps, lat 120.193 ms stddev 16.039 2024-04-25T00:09:17.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:16 smithi110 ceph-mon[29187]: pgmap v568: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 853 op/s 2024-04-25T00:09:17.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:16 smithi173 ceph-mon[32402]: pgmap v568: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 853 op/s 2024-04-25T00:09:17.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:16 smithi188 ceph-mon[33285]: pgmap v568: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 853 op/s 2024-04-25T00:09:20.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:19 smithi188 ceph-mon[33285]: pgmap v569: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:09:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:19 smithi110 ceph-mon[29187]: pgmap v569: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:09:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:19 smithi173 ceph-mon[32402]: pgmap v569: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:09:21.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 375.0 s, 823.2 tps, lat 121.292 ms stddev 34.957 2024-04-25T00:09:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:21 smithi188 ceph-mon[33285]: pgmap v570: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 856 op/s 2024-04-25T00:09:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:21 smithi110 ceph-mon[29187]: pgmap v570: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 856 op/s 2024-04-25T00:09:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:21 smithi173 ceph-mon[32402]: pgmap v570: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 856 op/s 2024-04-25T00:09:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 -- 172.21.15.110:0/352402836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b81006a0 msgr2=0x7f66b8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 --2- 172.21.15.110:0/352402836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b81006a0 0x7f66b8100ac0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009900 tx=0x7f66a802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 -- 172.21.15.110:0/352402836 shutdown_connections 2024-04-25T00:09:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 --2- 172.21.15.110:0/352402836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b8102250 0x7f66b8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 --2- 172.21.15.110:0/352402836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66b8101890 0x7f66b8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 --2- 172.21.15.110:0/352402836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b81006a0 0x7f66b8100ac0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 -- 172.21.15.110:0/352402836 >> 172.21.15.110:0/352402836 conn(0x7f66b80fbc10 msgr2=0x7f66b80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 -- 172.21.15.110:0/352402836 shutdown_connections 2024-04-25T00:09:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.490+0000 7f66bd75c640 1 -- 172.21.15.110:0/352402836 wait complete. 2024-04-25T00:09:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.491+0000 7f66bd75c640 1 Processor -- start 2024-04-25T00:09:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.491+0000 7f66bd75c640 1 -- start start 2024-04-25T00:09:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.491+0000 7f66bd75c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b81006a0 0x7f66b81979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66bd75c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66b8101890 0x7f66b8197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66bd75c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8102250 0x7f66b819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66bd75c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b810c1a0 con 0x7f66b81006a0 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66bd75c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66b810c020 con 0x7f66b8101890 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66bd75c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f66b810c320 con 0x7f66b8102250 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66b8101890 0x7f66b8197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66b8101890 0x7f66b8197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39368/0 (socket says 172.21.15.110:39368) 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b67fc640 1 -- 172.21.15.110:0/4041929803 learned_addr learned my addr 172.21.15.110:0/4041929803 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b81006a0 0x7f66b81979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b77fe640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8102250 0x7f66b819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b6ffd640 1 -- 172.21.15.110:0/4041929803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8102250 msgr2=0x7f66b819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8102250 0x7f66b819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b6ffd640 1 -- 172.21.15.110:0/4041929803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66b8101890 msgr2=0x7f66b8197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66b8101890 0x7f66b8197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b6ffd640 1 -- 172.21.15.110:0/4041929803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b819f5c0 con 0x7f66b81006a0 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b77fe640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8102250 0x7f66b819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b67fc640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66b8101890 0x7f66b8197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.492+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b81006a0 0x7f66b81979a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009230 tx=0x7f66a8031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.493+0000 7f669ffff640 1 -- 172.21.15.110:0/4041929803 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8035070 con 0x7f66b81006a0 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.493+0000 7f66bd75c640 1 -- 172.21.15.110:0/4041929803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b8102ab0 con 0x7f66b81006a0 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.493+0000 7f669ffff640 1 -- 172.21.15.110:0/4041929803 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66a803dc10 con 0x7f66b81006a0 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.493+0000 7f669ffff640 1 -- 172.21.15.110:0/4041929803 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8003d80 con 0x7f66b81006a0 2024-04-25T00:09:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.493+0000 7f66bd75c640 1 -- 172.21.15.110:0/4041929803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b8102fc0 con 0x7f66b81006a0 2024-04-25T00:09:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.496+0000 7f669ffff640 1 -- 172.21.15.110:0/4041929803 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f66a80035f0 con 0x7f66b81006a0 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.496+0000 7f66c045f740 1 Processor -- start 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.496+0000 7f66c045f740 1 -- start start 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66c045f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66c045f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c1840 0x556b1d6c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f669ffff640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66a00a2c30 0x7f66a00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66c045f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c69b0 0x556b1d6c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66c045f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b1d624cb0 con 0x556b1d6c0e50 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66c045f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556b1d5cd5b0 con 0x556b1d6c69b0 2024-04-25T00:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66c045f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b1d620c70 con 0x556b1d6c1840 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c1840 0x556b1d6c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6c1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43794/0 (socket says 172.21.15.110:43794) 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 -- 172.21.15.110:0/1175427331 learned_addr learned my addr 172.21.15.110:0/1175427331 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 --2- 172.21.15.110:0/1175427331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c69b0 0x556b1d6c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f669ffff640 1 -- 172.21.15.110:0/4041929803 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f66a80f3ba0 con 0x7f66b81006a0 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b67fc640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66a00a2c30 0x7f66a00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 -- 172.21.15.110:0/1175427331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c1840 msgr2=0x556b1d6c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 --2- 172.21.15.110:0/1175427331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c1840 0x556b1d6c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 -- 172.21.15.110:0/1175427331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c69b0 msgr2=0x556b1d6c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 --2- 172.21.15.110:0/1175427331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c69b0 0x556b1d6c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.497+0000 7f66b77fe640 1 -- 172.21.15.110:0/1175427331 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1d339210 con 0x556b1d6c0e50 2024-04-25T00:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.498+0000 7f66b67fc640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66a00a2c30 0x7f66a00a50f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0095f0 tx=0x7f66ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.500 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.498+0000 7f66b77fe640 1 --2- 172.21.15.110:0/1175427331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6c1270 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f66b0013f40 tx=0x7f66b0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792835b9191ef6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.498+0000 7f669d7fa640 1 -- 172.21.15.110:0/1175427331 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000fd10 con 0x556b1d6c0e50 2024-04-25T00:09:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.498+0000 7f669d7fa640 1 -- 172.21.15.110:0/1175427331 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b0013130 con 0x556b1d6c0e50 2024-04-25T00:09:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.498+0000 7f669d7fa640 1 -- 172.21.15.110:0/1175427331 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0004030 con 0x556b1d6c0e50 2024-04-25T00:09:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 -- 172.21.15.110:0/1175427331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 msgr2=0x556b1d6c1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 --2- 172.21.15.110:0/1175427331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6c1270 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f66b0013f40 tx=0x7f66b0009ee0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 -- 172.21.15.110:0/1175427331 shutdown_connections 2024-04-25T00:09:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 --2- 172.21.15.110:0/1175427331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c69b0 0x556b1d6c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 --2- 172.21.15.110:0/1175427331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c1840 0x556b1d6c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 --2- 172.21.15.110:0/1175427331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6c1270 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 -- 172.21.15.110:0/1175427331 >> 172.21.15.110:0/1175427331 conn(0x556b1d6bc4c0 msgr2=0x556b1d6bc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 -- 172.21.15.110:0/1175427331 shutdown_connections 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.499+0000 7f66c045f740 1 -- 172.21.15.110:0/1175427331 wait complete. 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.500+0000 7f66c045f740 1 -- start start 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.500+0000 7f66c045f740 1 Processor -- start 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.500+0000 7f66c045f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c0e50 0x556b1d6cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.500+0000 7f66c045f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c1840 0x556b1d6cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66c045f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c0e50 0x556b1d6cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c1840 0x556b1d6cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c0e50 0x556b1d6cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39386/0 (socket says 172.21.15.110:39386) 2024-04-25T00:09:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 -- 172.21.15.110:0/944226376 learned_addr learned my addr 172.21.15.110:0/944226376 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b1d6c9900 con 0x556b1d6c1840 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556b1d5e0b50 con 0x556b1d6c0e50 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b1d6c9a80 con 0x556b1d6c69b0 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66c045f740 10 client.0 _ll_register_callbacks cb 0x556b1d61bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b77fe640 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 -- 172.21.15.110:0/944226376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 msgr2=0x556b1d6d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 -- 172.21.15.110:0/944226376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c1840 msgr2=0x556b1d6cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c1840 0x556b1d6cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 -- 172.21.15.110:0/944226376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1d405620 con 0x556b1d6c0e50 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c0e50 0x556b1d6cb2e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66a8075c30 tx=0x7f66a8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f689ea41d6d7184 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f669aff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f669aff5640 1 -- 172.21.15.110:0/944226376 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80c0b30 con 0x556b1d6c0e50 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.501+0000 7f669aff5640 1 -- 172.21.15.110:0/944226376 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f66a80c0690 con 0x556b1d6c0e50 2024-04-25T00:09:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b0013130 con 0x556b1d6c0e50 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f669aff5640 1 -- 172.21.15.110:0/944226376 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66a80c0430 con 0x556b1d6c0e50 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 10 client.25424 fetch_fsmap learned FSMap version 32 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556b1d6d5bb0 con 0x556b1d6c0e50 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f669aff5640 1 -- 172.21.15.110:0/944226376 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b0013130 con 0x556b1d6c0e50 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f669aff5640 1 -- 172.21.15.110:0/944226376 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f66a80f1b90 con 0x556b1d6c0e50 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 10 client.25424 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 10 client.25424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 20 client.25424 populate_metadata read hostname 'smithi110' 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556b1d6d7290 0x556b1d6d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 4 client.25424 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.502+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1d6321f0 con 0x556b1d6d7290 2024-04-25T00:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.503+0000 7f66b67fc640 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556b1d6d7290 0x556b1d6d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.503+0000 7f669aff5640 10 client.25424 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.503+0000 7f66b67fc640 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556b1d6d7290 0x556b1d6d96e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20c297c10be845b1 server_cookie=d1f32c249fe4d7e5 in_seq=0 out_seq=0 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f669aff5640 1 -- 172.21.15.110:0/944226376 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b1d6321f0 con 0x556b1d6d7290 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f669aff5640 10 client.25424 handle_command_reply: tid=0 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 client.25424 shutdown 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c0e50 msgr2=0x556b1d6cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c0e50 0x556b1d6cb2e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66a8075c30 tx=0x7f66a8009260 comp rx=0 tx=0).stop 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 shutdown_connections 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556b1d6d7290 0x556b1d6d96e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c1840 0x556b1d6cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 --2- 172.21.15.110:0/944226376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c0e50 0x556b1d6cb2e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 >> 172.21.15.110:0/944226376 conn(0x556b1d6bc4c0 msgr2=0x556b1d6bb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f669aff5640 0 client.25424 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 shutdown_connections 2024-04-25T00:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 1 -- 172.21.15.110:0/944226376 wait complete. 2024-04-25T00:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.504+0000 7f66c045f740 20 client.25424 trim_cache size 0 max 16384 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66c045f740 1 Processor -- start 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66c045f740 1 -- start start 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66c045f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66c045f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66c045f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66c045f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81a9160 con 0x556b1d6c0e50 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66c045f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81a6a30 con 0x556b1d6c1840 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66c045f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81a4880 con 0x556b1d6c69b0 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43814/0 (socket says 172.21.15.110:43814) 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39400/0 (socket says 172.21.15.110:39400) 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.527+0000 7f66b6ffd640 1 -- 172.21.15.110:0/2806596346 learned_addr learned my addr 172.21.15.110:0/2806596346 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f66b67fc640 1 -- 172.21.15.110:0/2806596346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 msgr2=0x556b1d6cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f66b67fc640 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f66b67fc640 1 -- 172.21.15.110:0/2806596346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 msgr2=0x556b1d6caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f66b67fc640 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f66b67fc640 1 -- 172.21.15.110:0/2806596346 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1d6c35d0 con 0x556b1d6c1840 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f66b77fe640 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f66b67fc640 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d27f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x556b1d6d9f00 tx=0x7f66ac014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9afa15db457c0e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f669cff9640 1 -- 172.21.15.110:0/2806596346 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac02f070 con 0x556b1d6c1840 2024-04-25T00:09:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f669cff9640 1 -- 172.21.15.110:0/2806596346 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66ac02a470 con 0x556b1d6c1840 2024-04-25T00:09:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.528+0000 7f669cff9640 1 -- 172.21.15.110:0/2806596346 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac01cd60 con 0x556b1d6c1840 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 -- 172.21.15.110:0/2806596346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 msgr2=0x556b1d6d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d27f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x556b1d6d9f00 tx=0x7f66ac014000 comp rx=0 tx=0).stop 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 -- 172.21.15.110:0/2806596346 shutdown_connections 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d27f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 --2- 172.21.15.110:0/2806596346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 -- 172.21.15.110:0/2806596346 >> 172.21.15.110:0/2806596346 conn(0x556b1d6bc4c0 msgr2=0x556b1d6bc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 -- 172.21.15.110:0/2806596346 shutdown_connections 2024-04-25T00:09:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.529+0000 7f66c045f740 1 -- 172.21.15.110:0/2806596346 wait complete. 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 1 -- start start 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 1 Processor -- start 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81a4ef0 con 0x556b1d6c0e50 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81a6360 con 0x556b1d6c1840 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81a55c0 con 0x556b1d6c69b0 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66c045f740 10 client.0 _ll_register_callbacks cb 0x7f66a8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43822/0 (socket says 172.21.15.110:43822) 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66b6ffd640 1 -- 172.21.15.110:0/3821495077 learned_addr learned my addr 172.21.15.110:0/3821495077 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.530+0000 7f66b77fe640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b67fc640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b6ffd640 1 -- 172.21.15.110:0/3821495077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 msgr2=0x556b1d6d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b6ffd640 1 -- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 msgr2=0x556b1d6d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b6ffd640 1 -- 172.21.15.110:0/3821495077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1d6d16d0 con 0x556b1d6c0e50 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b77fe640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b67fc640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f66b6ffd640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6cfc30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f66a80bc890 tx=0x7f66a80314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55d257c171f17a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.531+0000 7f669aff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f669aff5640 1 -- 172.21.15.110:0/3821495077 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80a3060 con 0x556b1d6c0e50 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f669aff5640 1 -- 172.21.15.110:0/3821495077 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f66a80bcaa0 con 0x556b1d6c0e50 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 1 -- 172.21.15.110:0/3821495077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ac02a470 con 0x556b1d6c0e50 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f669aff5640 1 -- 172.21.15.110:0/3821495077 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66a80bc6d0 con 0x556b1d6c0e50 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 10 client.15846 fetch_fsmap learned FSMap version 32 2024-04-25T00:09:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 1 -- 172.21.15.110:0/3821495077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556b1d6d29e0 con 0x556b1d6c0e50 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f669aff5640 1 -- 172.21.15.110:0/3821495077 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66ac02a470 con 0x556b1d6c0e50 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f669aff5640 1 -- 172.21.15.110:0/3821495077 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f66a80ca020 con 0x556b1d6c0e50 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 10 client.15846 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 20 client.15846 populate_metadata read hostname 'smithi110' 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556b1d73dba0 0x556b1d73ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 4 client.15846 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66c045f740 1 -- 172.21.15.110:0/3821495077 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b1d62c540 con 0x556b1d73dba0 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.532+0000 7f66b67fc640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556b1d73dba0 0x556b1d73ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.533+0000 7f669aff5640 10 client.15846 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.533+0000 7f66b67fc640 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556b1d73dba0 0x556b1d73ffc0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f296e071547ac048 server_cookie=991a7774ccc9cf69 in_seq=0 out_seq=0 2024-04-25T00:09:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.552+0000 7f669aff5640 1 -- 172.21.15.110:0/3821495077 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b1d62c540 con 0x556b1d73dba0 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.552+0000 7f669aff5640 10 client.15846 handle_command_reply: tid=0 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.552+0000 7f66c045f740 1 client.15846 shutdown 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 -- 172.21.15.110:0/3821495077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 msgr2=0x556b1d6cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6cfc30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f66a80bc890 tx=0x7f66a80314a0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 -- 172.21.15.110:0/3821495077 shutdown_connections 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556b1d73dba0 0x556b1d73ffc0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b1d6c69b0 0x556b1d6d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b1d6c1840 0x556b1d6d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f669aff5640 0 client.15846 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 --2- 172.21.15.110:0/3821495077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b1d6c0e50 0x556b1d6cfc30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 -- 172.21.15.110:0/3821495077 >> 172.21.15.110:0/3821495077 conn(0x556b1d6bc4c0 msgr2=0x556b1d6caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 -- 172.21.15.110:0/3821495077 shutdown_connections 2024-04-25T00:09:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 1 -- 172.21.15.110:0/3821495077 wait complete. 2024-04-25T00:09:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.553+0000 7f66c045f740 20 client.15846 trim_cache size 0 max 16384 2024-04-25T00:09:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:09:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:09:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:09:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:09:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 -- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66a00a2c30 msgr2=0x7f66a00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66a00a2c30 0x7f66a00a50f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0095f0 tx=0x7f66ac005f90 comp rx=0 tx=0).stop 2024-04-25T00:09:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 -- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b81006a0 msgr2=0x7f66b81979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b81006a0 0x7f66b81979a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009230 tx=0x7f66a8031980 comp rx=0 tx=0).stop 2024-04-25T00:09:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 -- 172.21.15.110:0/4041929803 shutdown_connections 2024-04-25T00:09:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f66a00a2c30 0x7f66a00a50f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8102250 0x7f66b819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66b8101890 0x7f66b8197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 --2- 172.21.15.110:0/4041929803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b81006a0 0x7f66b81979a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 -- 172.21.15.110:0/4041929803 >> 172.21.15.110:0/4041929803 conn(0x7f66b80fbc10 msgr2=0x7f66b80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 -- 172.21.15.110:0/4041929803 shutdown_connections 2024-04-25T00:09:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:22.554+0000 7f66bd75c640 1 -- 172.21.15.110:0/4041929803 wait complete. 2024-04-25T00:09:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:23 smithi188 ceph-mon[33285]: pgmap v571: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-04-25T00:09:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:23 smithi110 ceph-mon[29187]: pgmap v571: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-04-25T00:09:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:23 smithi173 ceph-mon[32402]: pgmap v571: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-04-25T00:09:26.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:25 smithi188 ceph-mon[33285]: pgmap v572: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 922 op/s 2024-04-25T00:09:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:25 smithi110 ceph-mon[29187]: pgmap v572: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 922 op/s 2024-04-25T00:09:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:25 smithi173 ceph-mon[32402]: pgmap v572: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 922 op/s 2024-04-25T00:09:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 380.0 s, 717.8 tps, lat 138.895 ms stddev 30.988 2024-04-25T00:09:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:27 smithi188 ceph-mon[33285]: pgmap v573: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.8 MiB/s wr, 823 op/s 2024-04-25T00:09:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:27 smithi110 ceph-mon[29187]: pgmap v573: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.8 MiB/s wr, 823 op/s 2024-04-25T00:09:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:27 smithi173 ceph-mon[32402]: pgmap v573: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.8 MiB/s wr, 823 op/s 2024-04-25T00:09:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:29 smithi188 ceph-mon[33285]: pgmap v574: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.26k op/s 2024-04-25T00:09:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:29 smithi110 ceph-mon[29187]: pgmap v574: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.26k op/s 2024-04-25T00:09:30.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:29 smithi173 ceph-mon[32402]: pgmap v574: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.26k op/s 2024-04-25T00:09:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 385.0 s, 671.0 tps, lat 148.989 ms stddev 20.096 2024-04-25T00:09:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:31 smithi188 ceph-mon[33285]: pgmap v575: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:09:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:31 smithi110 ceph-mon[29187]: pgmap v575: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:09:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:31 smithi173 ceph-mon[32402]: pgmap v575: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:09:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:33 smithi188 ceph-mon[33285]: pgmap v576: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-04-25T00:09:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:33 smithi110 ceph-mon[29187]: pgmap v576: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-04-25T00:09:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:33 smithi173 ceph-mon[32402]: pgmap v576: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-04-25T00:09:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.691+0000 7f7d2a876640 1 -- 172.21.15.110:0/3545443765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d2410de50 msgr2=0x7f7d24110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.691+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3545443765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d2410de50 0x7f7d24110310 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009ca0 tx=0x7f7d1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.691+0000 7f7d2a876640 1 -- 172.21.15.110:0/3545443765 shutdown_connections 2024-04-25T00:09:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.691+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3545443765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d2410de50 0x7f7d24110310 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.691+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3545443765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d2410b370 0x7f7d2410d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.691+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3545443765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d241089b0 0x7f7d2410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.691+0000 7f7d2a876640 1 -- 172.21.15.110:0/3545443765 >> 172.21.15.110:0/3545443765 conn(0x7f7d2406e7b0 msgr2=0x7f7d24070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.692+0000 7f7d2a876640 1 -- 172.21.15.110:0/3545443765 shutdown_connections 2024-04-25T00:09:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.692+0000 7f7d2a876640 1 -- 172.21.15.110:0/3545443765 wait complete. 2024-04-25T00:09:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.693+0000 7f7d2a876640 1 Processor -- start 2024-04-25T00:09:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.693+0000 7f7d2a876640 1 -- start start 2024-04-25T00:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.693+0000 7f7d2a876640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d241089b0 0x7f7d2419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d2a876640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d2410b370 0x7f7d241a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d2a876640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d2410de50 0x7f7d241a7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d241089b0 0x7f7d2419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d241089b0 0x7f7d2419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:56776/0 (socket says 172.21.15.110:56776) 2024-04-25T00:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d28e13640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d2410de50 0x7f7d241a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d28e13640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d2410de50 0x7f7d241a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44914/0 (socket says 172.21.15.110:44914) 2024-04-25T00:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d1bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d2410b370 0x7f7d241a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d1bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d2410b370 0x7f7d241a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:57942/0 (socket says 172.21.15.110:57942) 2024-04-25T00:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d23fff640 1 -- 172.21.15.110:0/3862726436 learned_addr learned my addr 172.21.15.110:0/3862726436 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24115ae0 con 0x7f7d2410de50 2024-04-25T00:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.694+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24115960 con 0x7f7d2410b370 2024-04-25T00:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.695+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24115c60 con 0x7f7d241089b0 2024-04-25T00:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.695+0000 7f7d23fff640 1 -- 172.21.15.110:0/3862726436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d2410b370 msgr2=0x7f7d241a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.695+0000 7f7d23fff640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d2410b370 0x7f7d241a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.695+0000 7f7d23fff640 1 -- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d2410de50 msgr2=0x7f7d241a7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.695+0000 7f7d23fff640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d2410de50 0x7f7d241a7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.695+0000 7f7d23fff640 1 -- 172.21.15.110:0/3862726436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d241a7b00 con 0x7f7d241089b0 2024-04-25T00:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.695+0000 7f7d23fff640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d241089b0 0x7f7d2419fee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00e9c0 tx=0x7f7d0c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.696+0000 7f7d21ffb640 1 -- 172.21.15.110:0/3862726436 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00cd80 con 0x7f7d241089b0 2024-04-25T00:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.696+0000 7f7d21ffb640 1 -- 172.21.15.110:0/3862726436 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d0c0054d0 con 0x7f7d241089b0 2024-04-25T00:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.697+0000 7f7d21ffb640 1 -- 172.21.15.110:0/3862726436 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c010430 con 0x7f7d241089b0 2024-04-25T00:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.697+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d241a7df0 con 0x7f7d241089b0 2024-04-25T00:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.697+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d241a8250 con 0x7f7d241089b0 2024-04-25T00:09:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.700+0000 7f7d21ffb640 1 -- 172.21.15.110:0/3862726436 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7d0c004fb0 con 0x7f7d241089b0 2024-04-25T00:09:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.700+0000 7f7d2d586740 1 Processor -- start 2024-04-25T00:09:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.700+0000 7f7d2d586740 1 -- start start 2024-04-25T00:09:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee8e50 0x563150ee9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150ee9840 0x563150eee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150eee9b0 0x563150ef0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d2d586740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563150e4ccb0 con 0x563150eee9b0 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d2d586740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563150df55b0 con 0x563150ee8e50 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d2d586740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563150e48c70 con 0x563150ee9840 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d28e13640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee8e50 0x563150ee9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150ee9840 0x563150eee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150ee9840 0x563150eee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:56782/0 (socket says 172.21.15.110:56782) 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d23fff640 1 -- 172.21.15.110:0/1317230125 learned_addr learned my addr 172.21.15.110:0/1317230125 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/1317230125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150eee9b0 0x563150ef0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d21ffb640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7d100a2df0 0x7f7d100a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d23fff640 1 -- 172.21.15.110:0/1317230125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee8e50 msgr2=0x563150ee9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d23fff640 1 --2- 172.21.15.110:0/1317230125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee8e50 0x563150ee9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d23fff640 1 -- 172.21.15.110:0/1317230125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150eee9b0 msgr2=0x563150ef0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d23fff640 1 --2- 172.21.15.110:0/1317230125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150eee9b0 0x563150ef0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.701+0000 7f7d23fff640 1 -- 172.21.15.110:0/1317230125 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563150b61210 con 0x563150ee9840 2024-04-25T00:09:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.702+0000 7f7d21ffb640 1 -- 172.21.15.110:0/3862726436 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7d0c0cf360 con 0x7f7d241089b0 2024-04-25T00:09:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.702+0000 7f7d28e13640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7d100a2df0 0x7f7d100a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.702+0000 7f7d23fff640 1 --2- 172.21.15.110:0/1317230125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150ee9840 0x563150eee340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c06d0d0 tx=0x7f7d0c06dc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38951bb4a87aa1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.703+0000 7f7d1a7fc640 1 -- 172.21.15.110:0/1317230125 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c09b9c0 con 0x563150ee9840 2024-04-25T00:09:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.703+0000 7f7d28e13640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7d100a2df0 0x7f7d100a52b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x563150eea0a0 tx=0x7f7d1c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.703+0000 7f7d1a7fc640 1 -- 172.21.15.110:0/1317230125 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d0c06dcf0 con 0x563150ee9840 2024-04-25T00:09:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.703+0000 7f7d1a7fc640 1 -- 172.21.15.110:0/1317230125 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c06d160 con 0x563150ee9840 2024-04-25T00:09:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 -- 172.21.15.110:0/1317230125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150ee9840 msgr2=0x563150eee340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1317230125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150ee9840 0x563150eee340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c06d0d0 tx=0x7f7d0c06dc80 comp rx=0 tx=0).stop 2024-04-25T00:09:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 -- 172.21.15.110:0/1317230125 shutdown_connections 2024-04-25T00:09:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1317230125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150eee9b0 0x563150ef0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1317230125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150ee9840 0x563150eee340 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1317230125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee8e50 0x563150ee9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 -- 172.21.15.110:0/1317230125 >> 172.21.15.110:0/1317230125 conn(0x563150ee44c0 msgr2=0x563150ee48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 -- 172.21.15.110:0/1317230125 shutdown_connections 2024-04-25T00:09:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.704+0000 7f7d2d586740 1 -- 172.21.15.110:0/1317230125 wait complete. 2024-04-25T00:09:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.705+0000 7f7d2d586740 1 -- start start 2024-04-25T00:09:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.705+0000 7f7d2d586740 1 Processor -- start 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.705+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150eeb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.705+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.705+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.705+0000 7f7d2d586740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563150ef5d10 con 0x563150ee8e50 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.705+0000 7f7d2d586740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563150e08b50 con 0x563150ee9840 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.706+0000 7f7d2d586740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563150ef5e90 con 0x563150eee9b0 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.706+0000 7f7d2d586740 10 client.0 _ll_register_callbacks cb 0x563150e43ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.706+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150eeb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.706+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150eeb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44936/0 (socket says 172.21.15.110:44936) 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.706+0000 7f7d23fff640 1 -- 172.21.15.110:0/3112940051 learned_addr learned my addr 172.21.15.110:0/3112940051 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.706+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.706+0000 7f7d28e13640 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d1bfff640 1 -- 172.21.15.110:0/3112940051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 msgr2=0x563150eecd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eecd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d1bfff640 1 -- 172.21.15.110:0/3112940051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 msgr2=0x563150eeb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150eeb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d1bfff640 1 -- 172.21.15.110:0/3112940051 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563150c2d620 con 0x563150ee9840 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d28e13640 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eebd30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7d241a13f0 tx=0x7f7d140128e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edabdef526c997bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d037fe640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d037fe640 1 -- 172.21.15.110:0/3112940051 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14004030 con 0x563150ee9840 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d2d586740 1 -- 172.21.15.110:0/3112940051 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563150e5a1f0 con 0x563150ee9840 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.707+0000 7f7d037fe640 1 -- 172.21.15.110:0/3112940051 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7d140041d0 con 0x563150ee9840 2024-04-25T00:09:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d037fe640 1 -- 172.21.15.110:0/3112940051 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7d14023070 con 0x563150ee9840 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d2d586740 10 client.25442 fetch_fsmap learned FSMap version 32 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d2d586740 1 -- 172.21.15.110:0/3112940051 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563150f06390 con 0x563150ee9840 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d037fe640 1 -- 172.21.15.110:0/3112940051 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563150e5a1f0 con 0x563150ee9840 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d037fe640 1 -- 172.21.15.110:0/3112940051 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7d1400f040 con 0x563150ee9840 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d2d586740 10 client.25442 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d2d586740 10 client.25442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d2d586740 20 client.25442 populate_metadata read hostname 'smithi110' 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d2d586740 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563150f079b0 0x563150f09e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d2d586740 4 client.25442 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.708+0000 7f7d2d586740 1 -- 172.21.15.110:0/3112940051 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563150e54540 con 0x563150f079b0 2024-04-25T00:09:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.709+0000 7f7d23fff640 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563150f079b0 0x563150f09e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.709+0000 7f7d23fff640 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563150f079b0 0x563150f09e20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835e088f1794cad server_cookie=9a874750e49b7a77 in_seq=0 out_seq=0 2024-04-25T00:09:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.709+0000 7f7d037fe640 10 client.25442 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.710+0000 7f7d037fe640 1 -- 172.21.15.110:0/3112940051 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563150e54540 con 0x563150f079b0 2024-04-25T00:09:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d037fe640 10 client.25442 handle_command_reply: tid=0 2024-04-25T00:09:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 client.25442 shutdown 2024-04-25T00:09:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 -- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 msgr2=0x563150eebd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eebd30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7d241a13f0 tx=0x7f7d140128e0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 -- 172.21.15.110:0/3112940051 shutdown_connections 2024-04-25T00:09:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563150f079b0 0x563150f09e20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eebd30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 --2- 172.21.15.110:0/3112940051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150eeb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d037fe640 0 client.25442 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 -- 172.21.15.110:0/3112940051 >> 172.21.15.110:0/3112940051 conn(0x563150ee44c0 msgr2=0x563150ee35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 -- 172.21.15.110:0/3112940051 shutdown_connections 2024-04-25T00:09:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.711+0000 7f7d2d586740 1 -- 172.21.15.110:0/3112940051 wait complete. 2024-04-25T00:09:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.712+0000 7f7d2d586740 20 client.25442 trim_cache size 0 max 16384 2024-04-25T00:09:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.739+0000 7f7d2d586740 1 Processor -- start 2024-04-25T00:09:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.739+0000 7f7d2d586740 1 -- start start 2024-04-25T00:09:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.739+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150ee3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.739+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.739+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eea730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.739+0000 7f7d2d586740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241b1730 con 0x563150ee8e50 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.739+0000 7f7d2d586740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241af000 con 0x563150ee9840 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.739+0000 7f7d2d586740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241ace50 con 0x563150eee9b0 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eea730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:56810/0 (socket says 172.21.15.110:56810) 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 -- 172.21.15.110:0/2549614732 learned_addr learned my addr 172.21.15.110:0/2549614732 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d23fff640 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150ee3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 -- 172.21.15.110:0/2549614732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 msgr2=0x563150eea1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eea1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 -- 172.21.15.110:0/2549614732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 msgr2=0x563150ee3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150ee3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 -- 172.21.15.110:0/2549614732 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563150ee5230 con 0x563150eee9b0 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.740+0000 7f7d28e13640 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eea730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c031f40 tx=0x7f7d1c044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71151847fb87990d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.741+0000 7f7d19ffb640 1 -- 172.21.15.110:0/2549614732 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c003b70 con 0x563150eee9b0 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.741+0000 7f7d19ffb640 1 -- 172.21.15.110:0/2549614732 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d1c036040 con 0x563150eee9b0 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.741+0000 7f7d19ffb640 1 -- 172.21.15.110:0/2549614732 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c030820 con 0x563150eee9b0 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 -- 172.21.15.110:0/2549614732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 msgr2=0x563150eea730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eea730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c031f40 tx=0x7f7d1c044000 comp rx=0 tx=0).stop 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 -- 172.21.15.110:0/2549614732 shutdown_connections 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150eea730 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150eea1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 --2- 172.21.15.110:0/2549614732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150ee3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 -- 172.21.15.110:0/2549614732 >> 172.21.15.110:0/2549614732 conn(0x563150ee44c0 msgr2=0x563150ee2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 -- 172.21.15.110:0/2549614732 shutdown_connections 2024-04-25T00:09:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.742+0000 7f7d2d586740 1 -- 172.21.15.110:0/2549614732 wait complete. 2024-04-25T00:09:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.743+0000 7f7d2d586740 1 -- start start 2024-04-25T00:09:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.743+0000 7f7d2d586740 1 Processor -- start 2024-04-25T00:09:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150f01970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150f01eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d2d586740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150f02e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d2d586740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241ad4c0 con 0x563150ee8e50 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d2d586740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241ae930 con 0x563150ee9840 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d2d586740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241adb90 con 0x563150eee9b0 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d2d586740 10 client.0 _ll_register_callbacks cb 0x7f7d140098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d1bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150f01eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d23fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150f01970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d1bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150f01eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:57978/0 (socket says 172.21.15.110:57978) 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d1bfff640 1 -- 172.21.15.110:0/1363021807 learned_addr learned my addr 172.21.15.110:0/1363021807 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d28e13640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150f02e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d1bfff640 1 -- 172.21.15.110:0/1363021807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 msgr2=0x563150f02e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150f02e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d1bfff640 1 -- 172.21.15.110:0/1363021807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 msgr2=0x563150f01970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150f01970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.744+0000 7f7d1bfff640 1 -- 172.21.15.110:0/1363021807 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563150f03410 con 0x563150ee9840 2024-04-25T00:09:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.745+0000 7f7d1bfff640 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150f01eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d14003a30 tx=0x7f7d14003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b225dee66005260e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.745+0000 7f7d037fe640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:09:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.745+0000 7f7d037fe640 1 -- 172.21.15.110:0/1363021807 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14005280 con 0x563150ee9840 2024-04-25T00:09:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.746+0000 7f7d037fe640 1 -- 172.21.15.110:0/1363021807 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7d14005420 con 0x563150ee9840 2024-04-25T00:09:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.746+0000 7f7d2d586740 1 -- 172.21.15.110:0/1363021807 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1c036040 con 0x563150ee9840 2024-04-25T00:09:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.747+0000 7f7d037fe640 1 -- 172.21.15.110:0/1363021807 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7d14011710 con 0x563150ee9840 2024-04-25T00:09:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.747+0000 7f7d2d586740 10 client.25448 fetch_fsmap learned FSMap version 32 2024-04-25T00:09:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.747+0000 7f7d2d586740 1 -- 172.21.15.110:0/1363021807 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563150f047c0 con 0x563150ee9840 2024-04-25T00:09:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.747+0000 7f7d037fe640 1 -- 172.21.15.110:0/1363021807 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d1c036040 con 0x563150ee9840 2024-04-25T00:09:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.748+0000 7f7d037fe640 1 -- 172.21.15.110:0/1363021807 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7d1401c800 con 0x563150ee9840 2024-04-25T00:09:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.748+0000 7f7d2d586740 10 client.25448 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:09:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.748+0000 7f7d2d586740 10 client.25448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:09:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.748+0000 7f7d2d586740 20 client.25448 populate_metadata read hostname 'smithi110' 2024-04-25T00:09:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.748+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563150f6e5a0 0x563150f709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.748+0000 7f7d2d586740 4 client.25448 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:09:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.748+0000 7f7d2d586740 1 -- 172.21.15.110:0/1363021807 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563150dd28e0 con 0x563150f6e5a0 2024-04-25T00:09:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.748+0000 7f7d23fff640 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563150f6e5a0 0x563150f709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.749+0000 7f7d23fff640 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563150f6e5a0 0x563150f709c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbec01ca5726961c server_cookie=82c2f0300d444f8c in_seq=0 out_seq=0 2024-04-25T00:09:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.749+0000 7f7d037fe640 10 client.25448 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.780+0000 7f7d037fe640 1 -- 172.21.15.110:0/1363021807 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563150dd28e0 con 0x563150f6e5a0 2024-04-25T00:09:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.780+0000 7f7d037fe640 10 client.25448 handle_command_reply: tid=0 2024-04-25T00:09:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.780+0000 7f7d2d586740 1 client.25448 shutdown 2024-04-25T00:09:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.781+0000 7f7d2d586740 1 -- 172.21.15.110:0/1363021807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 msgr2=0x563150f01eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.781+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150f01eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d14003a30 tx=0x7f7d14003820 comp rx=0 tx=0).stop 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 1 -- 172.21.15.110:0/1363021807 shutdown_connections 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563150f6e5a0 0x563150f709c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563150eee9b0 0x563150f02e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563150ee9840 0x563150f01eb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 1 --2- 172.21.15.110:0/1363021807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563150ee8e50 0x563150f01970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 1 -- 172.21.15.110:0/1363021807 >> 172.21.15.110:0/1363021807 conn(0x563150ee44c0 msgr2=0x563150ee35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d037fe640 0 client.25448 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 1 -- 172.21.15.110:0/1363021807 shutdown_connections 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 1 -- 172.21.15.110:0/1363021807 wait complete. 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.782+0000 7f7d2d586740 20 client.25448 trim_cache size 0 max 16384 2024-04-25T00:09:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:09:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:09:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:09:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:09:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7d100a2df0 msgr2=0x7f7d100a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7d100a2df0 0x7f7d100a52b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x563150eea0a0 tx=0x7f7d1c005fd0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d241089b0 msgr2=0x7f7d2419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d241089b0 0x7f7d2419fee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00e9c0 tx=0x7f7d0c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 shutdown_connections 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7d100a2df0 0x7f7d100a52b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d2410de50 0x7f7d241a7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d2410b370 0x7f7d241a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 --2- 172.21.15.110:0/3862726436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d241089b0 0x7f7d2419fee0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 >> 172.21.15.110:0/3862726436 conn(0x7f7d2406e7b0 msgr2=0x7f7d2410bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 shutdown_connections 2024-04-25T00:09:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:35.784+0000 7f7d2a876640 1 -- 172.21.15.110:0/3862726436 wait complete. 2024-04-25T00:09:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:35 smithi188 ceph-mon[33285]: pgmap v577: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-04-25T00:09:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:35 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:35 smithi110 ceph-mon[29187]: pgmap v577: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-04-25T00:09:36.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:35 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:35 smithi173 ceph-mon[32402]: pgmap v577: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-04-25T00:09:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:35 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 390.0 s, 665.8 tps, lat 150.256 ms stddev 26.092 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:09:37.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:37.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:37.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:09:37.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:37.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:09:38.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:37 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-25T00:09:38.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:37 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-04-25T00:09:38.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:37 smithi188 ceph-mon[33285]: Unable to set osd_memory_target on smithi173 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T00:09:38.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:37 smithi188 ceph-mon[33285]: Unable to set osd_memory_target on smithi188 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:09:38.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:37 smithi188 ceph-mon[33285]: pgmap v578: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-04-25T00:09:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:37 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-25T00:09:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:37 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-04-25T00:09:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:37 smithi110 ceph-mon[29187]: Unable to set osd_memory_target on smithi173 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T00:09:38.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:37 smithi110 ceph-mon[29187]: Unable to set osd_memory_target on smithi188 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:09:38.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:37 smithi110 ceph-mon[29187]: pgmap v578: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-04-25T00:09:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:37 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-25T00:09:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:37 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-04-25T00:09:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:37 smithi173 ceph-mon[32402]: Unable to set osd_memory_target on smithi173 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-25T00:09:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:37 smithi173 ceph-mon[32402]: Unable to set osd_memory_target on smithi188 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:09:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:37 smithi173 ceph-mon[32402]: pgmap v578: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-04-25T00:09:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:39 smithi188 ceph-mon[33285]: pgmap v579: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-04-25T00:09:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:39 smithi110 ceph-mon[29187]: pgmap v579: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-04-25T00:09:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:39 smithi173 ceph-mon[32402]: pgmap v579: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-04-25T00:09:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 395.0 s, 673.0 tps, lat 146.775 ms stddev 30.755 2024-04-25T00:09:42.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:41 smithi188 ceph-mon[33285]: pgmap v580: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-04-25T00:09:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:41 smithi110 ceph-mon[29187]: pgmap v580: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-04-25T00:09:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:41 smithi173 ceph-mon[32402]: pgmap v580: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-04-25T00:09:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:43 smithi188 ceph-mon[33285]: pgmap v581: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-04-25T00:09:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:43 smithi110 ceph-mon[29187]: pgmap v581: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-04-25T00:09:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:43 smithi173 ceph-mon[32402]: pgmap v581: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-04-25T00:09:46.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:45 smithi188 ceph-mon[33285]: pgmap v582: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-04-25T00:09:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:45 smithi110 ceph-mon[29187]: pgmap v582: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-04-25T00:09:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:45 smithi173 ceph-mon[32402]: pgmap v582: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-04-25T00:09:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 400.0 s, 803.8 tps, lat 126.418 ms stddev 23.660 2024-04-25T00:09:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:47 smithi188 ceph-mon[33285]: pgmap v583: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-04-25T00:09:48.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:47 smithi110 ceph-mon[29187]: pgmap v583: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-04-25T00:09:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:47 smithi173 ceph-mon[32402]: pgmap v583: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-04-25T00:09:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 -- 172.21.15.110:0/3789998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b01006a0 msgr2=0x7f14b0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 --2- 172.21.15.110:0/3789998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b01006a0 0x7f14b0100ac0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009900 tx=0x7f14a402a410 comp rx=0 tx=0).stop 2024-04-25T00:09:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 -- 172.21.15.110:0/3789998396 shutdown_connections 2024-04-25T00:09:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 --2- 172.21.15.110:0/3789998396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14b0102250 0x7f14b0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 --2- 172.21.15.110:0/3789998396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b0101890 0x7f14b0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 --2- 172.21.15.110:0/3789998396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b01006a0 0x7f14b0100ac0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 -- 172.21.15.110:0/3789998396 >> 172.21.15.110:0/3789998396 conn(0x7f14b00fbc10 msgr2=0x7f14b00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 -- 172.21.15.110:0/3789998396 shutdown_connections 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.919+0000 7f14b6dee640 1 -- 172.21.15.110:0/3789998396 wait complete. 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b6dee640 1 Processor -- start 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b6dee640 1 -- start start 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b6dee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b01006a0 0x7f14b0195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b6dee640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14b0101890 0x7f14b0195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b6dee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0102250 0x7f14b019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b6dee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010b120 con 0x7f14b0102250 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b6dee640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010afa0 con 0x7f14b0101890 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b6dee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14b010b2a0 con 0x7f14b01006a0 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14affff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14b0101890 0x7f14b0195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14affff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14b0101890 0x7f14b0195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:51194/0 (socket says 172.21.15.110:51194) 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b4b8a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b01006a0 0x7f14b0195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14affff640 1 -- 172.21.15.110:0/1705444118 learned_addr learned my addr 172.21.15.110:0/1705444118 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.920+0000 7f14b538b640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0102250 0x7f14b019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14affff640 1 -- 172.21.15.110:0/1705444118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b01006a0 msgr2=0x7f14b0195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14affff640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b01006a0 0x7f14b0195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14affff640 1 -- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0102250 msgr2=0x7f14b019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14affff640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0102250 0x7f14b019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14affff640 1 -- 172.21.15.110:0/1705444118 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b019d390 con 0x7f14b0101890 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14affff640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14b0101890 0x7f14b0195cb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e9f0 tx=0x7f149c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14adffb640 1 -- 172.21.15.110:0/1705444118 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c00cd80 con 0x7f14b0101890 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14b6dee640 1 -- 172.21.15.110:0/1705444118 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b019d680 con 0x7f14b0101890 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14adffb640 1 -- 172.21.15.110:0/1705444118 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f149c0054d0 con 0x7f14b0101890 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14adffb640 1 -- 172.21.15.110:0/1705444118 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c010490 con 0x7f14b0101890 2024-04-25T00:09:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.921+0000 7f14b6dee640 1 -- 172.21.15.110:0/1705444118 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b019dbd0 con 0x7f14b0101890 2024-04-25T00:09:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.924+0000 7f14adffb640 1 -- 172.21.15.110:0/1705444118 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f149c005020 con 0x7f14b0101890 2024-04-25T00:09:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.924+0000 7f14b9bd9740 1 Processor -- start 2024-04-25T00:09:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.924+0000 7f14b9bd9740 1 -- start start 2024-04-25T00:09:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14adffb640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f14a00a4e40 0x7f14a00a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40ac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5557e400acb0 con 0x5557e40ac9b0 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5557e3fb35b0 con 0x5557e40a6e50 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5557e4006c70 con 0x5557e40a7840 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38702/0 (socket says 172.21.15.110:38702) 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 -- 172.21.15.110:0/2100137753 learned_addr learned my addr 172.21.15.110:0/2100137753 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f14a00a4e40 0x7f14a00a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14adffb640 1 -- 172.21.15.110:0/1705444118 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f149c0cf3d0 con 0x7f14b0101890 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14affff640 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 -- 172.21.15.110:0/2100137753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 msgr2=0x5557e40ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40ac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 -- 172.21.15.110:0/2100137753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 msgr2=0x5557e40a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 -- 172.21.15.110:0/2100137753 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e3d1f210 con 0x5557e40ac9b0 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.925+0000 7f14b538b640 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40aee40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14a8011e20 tx=0x7f14a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fb5cf83b3cc401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.926+0000 7f14b538b640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f14a00a4e40 0x7f14a00a7300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f14a800fc90 tx=0x7f14a8018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.926+0000 7f149affd640 1 -- 172.21.15.110:0/2100137753 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8022070 con 0x5557e40ac9b0 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.926+0000 7f149affd640 1 -- 172.21.15.110:0/2100137753 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14a8009e10 con 0x5557e40ac9b0 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.926+0000 7f149affd640 1 -- 172.21.15.110:0/2100137753 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8004290 con 0x5557e40ac9b0 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 -- 172.21.15.110:0/2100137753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 msgr2=0x5557e40aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40aee40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14a8011e20 tx=0x7f14a800f790 comp rx=0 tx=0).stop 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 -- 172.21.15.110:0/2100137753 shutdown_connections 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40aee40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/2100137753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 -- 172.21.15.110:0/2100137753 >> 172.21.15.110:0/2100137753 conn(0x5557e40a24c0 msgr2=0x5557e40a28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 -- 172.21.15.110:0/2100137753 shutdown_connections 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 -- 172.21.15.110:0/2100137753 wait complete. 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 -- start start 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.927+0000 7f14b9bd9740 1 Processor -- start 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5557e40af900 con 0x5557e40ac9b0 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5557e3fc6b50 con 0x5557e40a6e50 2024-04-25T00:09:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5557e40afa80 con 0x5557e40a7840 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b9bd9740 10 client.0 _ll_register_callbacks cb 0x5557e4001ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b538b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b538b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38706/0 (socket says 172.21.15.110:38706) 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b538b640 1 -- 172.21.15.110:0/1123430800 learned_addr learned my addr 172.21.15.110:0/1123430800 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14affff640 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14affff640 1 -- 172.21.15.110:0/1123430800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 msgr2=0x5557e40b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14affff640 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14affff640 1 -- 172.21.15.110:0/1123430800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 msgr2=0x5557e40b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14affff640 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14affff640 1 -- 172.21.15.110:0/1123430800 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e3deb620 con 0x5557e40a7840 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.928+0000 7f14b538b640 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.929+0000 7f14affff640 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b17e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f149c00c6a0 tx=0x7f149c00d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43c095c1224e58e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.929+0000 7f148ffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.929+0000 7f148ffff640 1 -- 172.21.15.110:0/1123430800 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c09c300 con 0x5557e40a7840 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.929+0000 7f14b9bd9740 1 -- 172.21.15.110:0/1123430800 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5557e40181f0 con 0x5557e40a7840 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.929+0000 7f148ffff640 1 -- 172.21.15.110:0/1123430800 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f149c0046c0 con 0x5557e40a7840 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.929+0000 7f148ffff640 1 -- 172.21.15.110:0/1123430800 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f149c0120f0 con 0x5557e40a7840 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.929+0000 7f14b9bd9740 10 client.25432 fetch_fsmap learned FSMap version 32 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.929+0000 7f14b9bd9740 1 -- 172.21.15.110:0/1123430800 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5557e40bfff0 con 0x5557e40a7840 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f148ffff640 1 -- 172.21.15.110:0/1123430800 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5557e40181f0 con 0x5557e40a7840 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f148ffff640 1 -- 172.21.15.110:0/1123430800 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f149c0a43e0 con 0x5557e40a7840 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f14b9bd9740 10 client.25432 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f14b9bd9740 10 client.25432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f14b9bd9740 20 client.25432 populate_metadata read hostname 'smithi110' 2024-04-25T00:09:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5557e40c1840 0x5557e40c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f14b9bd9740 4 client.25432 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:09:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f14b9bd9740 1 -- 172.21.15.110:0/1123430800 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557e4012540 con 0x5557e40c1840 2024-04-25T00:09:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.930+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5557e40c1840 0x5557e40c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.931+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5557e40c1840 0x5557e40c3c30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cae3f8e34e457b38 server_cookie=783e8d124447026f in_seq=0 out_seq=0 2024-04-25T00:09:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.931+0000 7f148ffff640 10 client.25432 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.931+0000 7f148ffff640 1 -- 172.21.15.110:0/1123430800 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557e4012540 con 0x5557e40c1840 2024-04-25T00:09:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.931+0000 7f148ffff640 10 client.25432 handle_command_reply: tid=0 2024-04-25T00:09:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.931+0000 7f14b9bd9740 1 client.25432 shutdown 2024-04-25T00:09:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 -- 172.21.15.110:0/1123430800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 msgr2=0x5557e40b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b17e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f149c00c6a0 tx=0x7f149c00d340 comp rx=0 tx=0).stop 2024-04-25T00:09:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 -- 172.21.15.110:0/1123430800 shutdown_connections 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5557e40c1840 0x5557e40c3c30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40ac9b0 0x5557e40b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b17e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/1123430800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40a6e50 0x5557e40b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 -- 172.21.15.110:0/1123430800 >> 172.21.15.110:0/1123430800 conn(0x5557e40a24c0 msgr2=0x5557e40a15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f148ffff640 0 client.25432 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 -- 172.21.15.110:0/1123430800 shutdown_connections 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 1 -- 172.21.15.110:0/1123430800 wait complete. 2024-04-25T00:09:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.932+0000 7f14b9bd9740 20 client.25432 trim_cache size 0 max 16384 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.951+0000 7f14b9bd9740 1 Processor -- start 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.951+0000 7f14b9bd9740 1 -- start start 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.951+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.951+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.951+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40a1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.951+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a9180 con 0x5557e40a6e50 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.951+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a6a50 con 0x5557e40ac9b0 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.951+0000 7f14b9bd9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a48a0 con 0x5557e40a7840 2024-04-25T00:09:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b538b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b4b8a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b538b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:51224/0 (socket says 172.21.15.110:51224) 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b538b640 1 -- 172.21.15.110:0/614535505 learned_addr learned my addr 172.21.15.110:0/614535505 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14affff640 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b4b8a640 1 -- 172.21.15.110:0/614535505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 msgr2=0x5557e40b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b4b8a640 1 -- 172.21.15.110:0/614535505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 msgr2=0x5557e40a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b4b8a640 1 -- 172.21.15.110:0/614535505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e40a2060 con 0x5557e40a6e50 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b538b640 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40a1440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009230 tx=0x7f14a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ecc125dd4960d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f149a7fc640 1 -- 172.21.15.110:0/614535505 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4035040 con 0x5557e40a6e50 2024-04-25T00:09:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f149a7fc640 1 -- 172.21.15.110:0/614535505 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14a404c070 con 0x5557e40a6e50 2024-04-25T00:09:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.952+0000 7f149a7fc640 1 -- 172.21.15.110:0/614535505 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4031580 con 0x5557e40a6e50 2024-04-25T00:09:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.953+0000 7f14b9bd9740 1 -- 172.21.15.110:0/614535505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 msgr2=0x5557e40a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.953+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40a1440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009230 tx=0x7f14a4009260 comp rx=0 tx=0).stop 2024-04-25T00:09:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.953+0000 7f14b9bd9740 1 -- 172.21.15.110:0/614535505 shutdown_connections 2024-04-25T00:09:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.954+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.954+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.954+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/614535505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40a1440 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.954+0000 7f14b9bd9740 1 -- 172.21.15.110:0/614535505 >> 172.21.15.110:0/614535505 conn(0x5557e40a24c0 msgr2=0x5557e40a0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.954+0000 7f14b9bd9740 1 -- 172.21.15.110:0/614535505 shutdown_connections 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.954+0000 7f14b9bd9740 1 -- 172.21.15.110:0/614535505 wait complete. 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.954+0000 7f14b9bd9740 1 -- start start 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b9bd9740 1 Processor -- start 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b4b8a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b4b8a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38726/0 (socket says 172.21.15.110:38726) 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b9bd9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b4b8a640 1 -- 172.21.15.110:0/3125371580 learned_addr learned my addr 172.21.15.110:0/3125371580 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a4f10 con 0x5557e40a6e50 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a6380 con 0x5557e40ac9b0 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14b01a55e0 con 0x5557e40a7840 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.955+0000 7f14affff640 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.956+0000 7f14b538b640 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.956+0000 7f14b9bd9740 10 client.0 _ll_register_callbacks cb 0x7f149c004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.956+0000 7f14affff640 1 -- 172.21.15.110:0/3125371580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 msgr2=0x5557e40b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.956+0000 7f14affff640 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40b6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.956+0000 7f14affff640 1 -- 172.21.15.110:0/3125371580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 msgr2=0x5557e40b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.956+0000 7f14affff640 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.956+0000 7f14affff640 1 -- 172.21.15.110:0/3125371580 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e40b7410 con 0x5557e40a7840 2024-04-25T00:09:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.956+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:09:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.957+0000 7f14affff640 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b5ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f149c0a4340 tx=0x7f149c0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af051c43b4f376ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.957+0000 7f148ffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:09:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.957+0000 7f148ffff640 1 -- 172.21.15.110:0/3125371580 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c09c300 con 0x5557e40a7840 2024-04-25T00:09:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.957+0000 7f148ffff640 1 -- 172.21.15.110:0/3125371580 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f149c09b050 con 0x5557e40a7840 2024-04-25T00:09:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.957+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a404c070 con 0x5557e40a7840 2024-04-25T00:09:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.957+0000 7f148ffff640 1 -- 172.21.15.110:0/3125371580 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f149c09bc00 con 0x5557e40a7840 2024-04-25T00:09:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.957+0000 7f14b9bd9740 10 client.25438 fetch_fsmap learned FSMap version 32 2024-04-25T00:09:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.957+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5557e40b86e0 con 0x5557e40a7840 2024-04-25T00:09:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f148ffff640 1 -- 172.21.15.110:0/3125371580 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14a404c070 con 0x5557e40a7840 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f148ffff640 1 -- 172.21.15.110:0/3125371580 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f149c0a4b50 con 0x5557e40a7840 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f14b9bd9740 10 client.25438 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f14b9bd9740 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f14b9bd9740 20 client.25438 populate_metadata read hostname 'smithi110' 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5557e4127fc0 0x5557e412a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f14b9bd9740 4 client.25438 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557e3f908e0 con 0x5557e4127fc0 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5557e4127fc0 0x5557e412a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.958+0000 7f148ffff640 10 client.25438 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.959+0000 7f14b4b8a640 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5557e4127fc0 0x5557e412a3b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c13e1752a20660c server_cookie=c8ae2864a10678e3 in_seq=0 out_seq=0 2024-04-25T00:09:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f148ffff640 1 -- 172.21.15.110:0/3125371580 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557e3f908e0 con 0x5557e4127fc0 2024-04-25T00:09:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f148ffff640 10 client.25438 handle_command_reply: tid=0 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f14b9bd9740 1 client.25438 shutdown 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 msgr2=0x5557e40b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b5ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f149c0a4340 tx=0x7f149c0048f0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 shutdown_connections 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5557e4127fc0 0x5557e412a3b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5557e40ac9b0 0x5557e40b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557e40a7840 0x5557e40b5ef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.992+0000 7f14b9bd9740 1 --2- 172.21.15.110:0/3125371580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557e40a6e50 0x5557e40b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.993+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 >> 172.21.15.110:0/3125371580 conn(0x5557e40a24c0 msgr2=0x5557e40a15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.993+0000 7f148ffff640 0 client.25438 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.993+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 shutdown_connections 2024-04-25T00:09:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.993+0000 7f14b9bd9740 1 -- 172.21.15.110:0/3125371580 wait complete. 2024-04-25T00:09:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.993+0000 7f14b9bd9740 20 client.25438 trim_cache size 0 max 16384 2024-04-25T00:09:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:09:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:09:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:09:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:09:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 -- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f14a00a4e40 msgr2=0x7f14a00a7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f14a00a4e40 0x7f14a00a7300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f14a800fc90 tx=0x7f14a8018000 comp rx=0 tx=0).stop 2024-04-25T00:09:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 -- 172.21.15.110:0/1705444118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14b0101890 msgr2=0x7f14b0195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14b0101890 0x7f14b0195cb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e9f0 tx=0x7f149c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 -- 172.21.15.110:0/1705444118 shutdown_connections 2024-04-25T00:09:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f14a00a4e40 0x7f14a00a7300 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14b0102250 0x7f14b019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14b0101890 0x7f14b0195cb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 --2- 172.21.15.110:0/1705444118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14b01006a0 0x7f14b0195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.994+0000 7f14b6dee640 1 -- 172.21.15.110:0/1705444118 >> 172.21.15.110:0/1705444118 conn(0x7f14b00fbc10 msgr2=0x7f14b0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.995+0000 7f14b6dee640 1 -- 172.21.15.110:0/1705444118 shutdown_connections 2024-04-25T00:09:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:09:48.995+0000 7f14b6dee640 1 -- 172.21.15.110:0/1705444118 wait complete. 2024-04-25T00:09:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:49 smithi188 ceph-mon[33285]: pgmap v584: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-04-25T00:09:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:49 smithi110 ceph-mon[29187]: pgmap v584: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-04-25T00:09:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:49 smithi173 ceph-mon[32402]: pgmap v584: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-04-25T00:09:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 405.0 s, 831.0 tps, lat 120.294 ms stddev 23.296 2024-04-25T00:09:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:51 smithi188 ceph-mon[33285]: pgmap v585: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-04-25T00:09:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:51 smithi110 ceph-mon[29187]: pgmap v585: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-04-25T00:09:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:51 smithi173 ceph-mon[32402]: pgmap v585: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-04-25T00:09:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:53 smithi188 ceph-mon[33285]: pgmap v586: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-04-25T00:09:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:53 smithi110 ceph-mon[29187]: pgmap v586: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-04-25T00:09:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:53 smithi173 ceph-mon[32402]: pgmap v586: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-04-25T00:09:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:55 smithi188 ceph-mon[33285]: pgmap v587: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.23k op/s 2024-04-25T00:09:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:55 smithi110 ceph-mon[29187]: pgmap v587: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.23k op/s 2024-04-25T00:09:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:55 smithi173 ceph-mon[32402]: pgmap v587: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.23k op/s 2024-04-25T00:09:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 410.0 s, 835.6 tps, lat 119.298 ms stddev 10.395 2024-04-25T00:09:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:57 smithi188 ceph-mon[33285]: pgmap v588: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 981 op/s 2024-04-25T00:09:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:57 smithi110 ceph-mon[29187]: pgmap v588: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 981 op/s 2024-04-25T00:09:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:57 smithi173 ceph-mon[32402]: pgmap v588: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 981 op/s 2024-04-25T00:10:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:09:59 smithi188 ceph-mon[33285]: pgmap v589: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-04-25T00:10:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:09:59 smithi110 ceph-mon[29187]: pgmap v589: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-04-25T00:10:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:09:59 smithi173 ceph-mon[32402]: pgmap v589: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-04-25T00:10:01.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:00 smithi188 ceph-mon[33285]: overall HEALTH_OK 2024-04-25T00:10:01.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:00 smithi110 ceph-mon[29187]: overall HEALTH_OK 2024-04-25T00:10:01.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:00 smithi173 ceph-mon[32402]: overall HEALTH_OK 2024-04-25T00:10:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 415.0 s, 842.8 tps, lat 118.583 ms stddev 13.418 2024-04-25T00:10:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:01 smithi188 ceph-mon[33285]: pgmap v590: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.6 MiB/s wr, 908 op/s 2024-04-25T00:10:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:01 smithi110 ceph-mon[29187]: pgmap v590: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.6 MiB/s wr, 908 op/s 2024-04-25T00:10:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:01 smithi173 ceph-mon[32402]: pgmap v590: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.6 MiB/s wr, 908 op/s 2024-04-25T00:10:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.127+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/676071896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e141006a0 msgr2=0x7f3e14100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.127+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/676071896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e141006a0 0x7f3e14100ac0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009900 tx=0x7f3e1002a410 comp rx=0 tx=0).stop 2024-04-25T00:10:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.127+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/676071896 shutdown_connections 2024-04-25T00:10:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.127+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/676071896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e14102250 0x7f3e14108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.127+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/676071896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e14101890 0x7f3e14101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.127+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/676071896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e141006a0 0x7f3e14100ac0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.127+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/676071896 >> 172.21.15.110:0/676071896 conn(0x7f3e140fbc10 msgr2=0x7f3e140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.128+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/676071896 shutdown_connections 2024-04-25T00:10:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.128+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/676071896 wait complete. 2024-04-25T00:10:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e1b9d5640 1 Processor -- start 2024-04-25T00:10:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e1b9d5640 1 -- start start 2024-04-25T00:10:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e1b9d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e141006a0 0x7f3e14199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e1b9d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e14101890 0x7f3e1419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e1b9d5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e14102250 0x7f3e141a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e1b9d5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1410b120 con 0x7f3e14101890 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e1b9d5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1410afa0 con 0x7f3e14102250 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e1b9d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1410b2a0 con 0x7f3e141006a0 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e18f70640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e14101890 0x7f3e1419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e19771640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e141006a0 0x7f3e14199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e19771640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e141006a0 0x7f3e14199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49942/0 (socket says 172.21.15.110:49942) 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e18f70640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e14101890 0x7f3e1419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58406/0 (socket says 172.21.15.110:58406) 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e19771640 1 -- 172.21.15.110:0/3156816350 learned_addr learned my addr 172.21.15.110:0/3156816350 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.129+0000 7f3e19f72640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e14102250 0x7f3e141a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e18f70640 1 -- 172.21.15.110:0/3156816350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e141006a0 msgr2=0x7f3e14199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e18f70640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e141006a0 0x7f3e14199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e18f70640 1 -- 172.21.15.110:0/3156816350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e14102250 msgr2=0x7f3e141a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e18f70640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e14102250 0x7f3e141a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e18f70640 1 -- 172.21.15.110:0/3156816350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e141a1760 con 0x7f3e14101890 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e19771640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e141006a0 0x7f3e14199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e18f70640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e14101890 0x7f3e1419a080 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400e9f0 tx=0x7f3e0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e0a7fc640 1 -- 172.21.15.110:0/3156816350 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400cd80 con 0x7f3e14101890 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/3156816350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e141a1a50 con 0x7f3e14101890 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/3156816350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e141a1ff0 con 0x7f3e14101890 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.130+0000 7f3e0a7fc640 1 -- 172.21.15.110:0/3156816350 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e040035f0 con 0x7f3e14101890 2024-04-25T00:10:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.131+0000 7f3e0a7fc640 1 -- 172.21.15.110:0/3156816350 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04010690 con 0x7f3e14101890 2024-04-25T00:10:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e1e7c1740 1 Processor -- start 2024-04-25T00:10:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e0a7fc640 1 -- 172.21.15.110:0/3156816350 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3e040108f0 con 0x7f3e14101890 2024-04-25T00:10:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e1e7c1740 1 -- start start 2024-04-25T00:10:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262762e50 0x559262763270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x559262768340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592627689b0 0x55926276ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5592626c6cb0 con 0x559262763840 2024-04-25T00:10:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55926266f5b0 con 0x559262762e50 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5592626c2c70 con 0x5592627689b0 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e19771640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x559262768340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e19771640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x559262768340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58408/0 (socket says 172.21.15.110:58408) 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e19771640 1 -- 172.21.15.110:0/1826273126 learned_addr learned my addr 172.21.15.110:0/1826273126 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.134+0000 7f3e18f70640 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592627689b0 0x55926276ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e19771640 1 -- 172.21.15.110:0/1826273126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592627689b0 msgr2=0x55926276ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e19771640 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592627689b0 0x55926276ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e19771640 1 -- 172.21.15.110:0/1826273126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262762e50 msgr2=0x559262763270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e19f72640 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262762e50 0x559262763270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e19771640 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262762e50 0x559262763270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e19771640 1 -- 172.21.15.110:0/1826273126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592623db210 con 0x559262763840 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e18f70640 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592627689b0 0x55926276ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:10:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e0a7fc640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e000a2cb0 0x7f3e000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e19771640 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x559262768340 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3e1002a930 tx=0x7f3e10005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef03515fc9b5466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e0a7fc640 1 -- 172.21.15.110:0/3156816350 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3e040cf130 con 0x7f3e14101890 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3dff7fe640 1 -- 172.21.15.110:0/1826273126 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10035070 con 0x559262763840 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3e19f72640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e000a2cb0 0x7f3e000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3dff7fe640 1 -- 172.21.15.110:0/1826273126 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e10041b60 con 0x559262763840 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.135+0000 7f3dff7fe640 1 -- 172.21.15.110:0/1826273126 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100317e0 con 0x559262763840 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/1826273126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 msgr2=0x559262768340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x559262768340 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3e1002a930 tx=0x7f3e10005c00 comp rx=0 tx=0).stop 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e19f72640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e000a2cb0 0x7f3e000a5170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x5592627640a0 tx=0x7f3e0c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/1826273126 shutdown_connections 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592627689b0 0x55926276ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x559262768340 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/1826273126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262762e50 0x559262763270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/1826273126 >> 172.21.15.110:0/1826273126 conn(0x55926275e4c0 msgr2=0x55926275e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/1826273126 shutdown_connections 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.136+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/1826273126 wait complete. 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 1 -- start start 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 1 Processor -- start 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x55926276d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x55926276d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x5592627747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55926276b900 con 0x5592627689b0 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559262682b50 con 0x559262763840 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55926276ba80 con 0x559262762e50 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e1e7c1740 10 client.0 _ll_register_callbacks cb 0x5592626bdca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e18f70640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x55926276d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e18f70640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x55926276d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:53954/0 (socket says 172.21.15.110:53954) 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e19f72640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x5592627747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e18f70640 1 -- 172.21.15.110:0/593027329 learned_addr learned my addr 172.21.15.110:0/593027329 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.137+0000 7f3e19771640 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x55926276d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3e18f70640 1 -- 172.21.15.110:0/593027329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 msgr2=0x55926276d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3e18f70640 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x55926276d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3e18f70640 1 -- 172.21.15.110:0/593027329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 msgr2=0x5592627747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3e18f70640 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x5592627747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3e18f70640 1 -- 172.21.15.110:0/593027329 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592624a7620 con 0x559262763840 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3e19f72640 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x5592627747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3e18f70640 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x55926276d7b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e04098390 tx=0x7f3e0409c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f9420b44d90307f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3dfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:10:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3dfcff9640 1 -- 172.21.15.110:0/593027329 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e040b4060 con 0x559262763840 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3dfcff9640 1 -- 172.21.15.110:0/593027329 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3e04098030 con 0x559262763840 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.138+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/593027329 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e10041b60 con 0x559262763840 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3dfcff9640 1 -- 172.21.15.110:0/593027329 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e040a4630 con 0x559262763840 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e1e7c1740 10 client.25475 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/593027329 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559262777b40 con 0x559262763840 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3dfcff9640 1 -- 172.21.15.110:0/593027329 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e040ba170 con 0x559262763840 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3dfcff9640 1 -- 172.21.15.110:0/593027329 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f3e040b4bc0 con 0x559262763840 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e1e7c1740 10 client.25475 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e1e7c1740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e1e7c1740 20 client.25475 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559262779220 0x55926277b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e1e7c1740 4 client.25475 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/593027329 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3e10041b60 con 0x559262779220 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.139+0000 7f3e19771640 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559262779220 0x55926277b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.140+0000 7f3dfcff9640 10 client.25475 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.140+0000 7f3e19771640 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559262779220 0x55926277b680 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a050cee9b2f953 server_cookie=a9d439b4b0e2c711 in_seq=0 out_seq=0 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3dfcff9640 1 -- 172.21.15.110:0/593027329 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3e10041b60 con 0x559262779220 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3dfcff9640 10 client.25475 handle_command_reply: tid=0 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 client.25475 shutdown 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/593027329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 msgr2=0x55926276d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x55926276d7b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e04098390 tx=0x7f3e0409c0c0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/593027329 shutdown_connections 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559262779220 0x55926277b680 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x5592627747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x55926276d7b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/593027329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x55926276d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3dfcff9640 0 client.25475 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/593027329 >> 172.21.15.110:0/593027329 conn(0x55926275e4c0 msgr2=0x55926275d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/593027329 shutdown_connections 2024-04-25T00:10:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.141+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/593027329 wait complete. 2024-04-25T00:10:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.142+0000 7f3e1e7c1740 20 client.25475 trim_cache size 0 max 16384 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.163+0000 7f3e1e7c1740 1 Processor -- start 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.163+0000 7f3e1e7c1740 1 -- start start 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x559262764c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x5592627651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592627689b0 0x559262761520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8010b80 con 0x559262763840 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3df800e450 con 0x5592627689b0 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3df800c2a0 con 0x559262762e50 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19f72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592627689b0 0x559262761520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19f72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592627689b0 0x559262761520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:53964/0 (socket says 172.21.15.110:53964) 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e18f70640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x5592627651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19f72640 1 -- 172.21.15.110:0/523770510 learned_addr learned my addr 172.21.15.110:0/523770510 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19771640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x559262764c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19f72640 1 -- 172.21.15.110:0/523770510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 msgr2=0x559262764c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19f72640 1 --2- 172.21.15.110:0/523770510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x559262764c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19f72640 1 -- 172.21.15.110:0/523770510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 msgr2=0x5592627651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19f72640 1 --2- 172.21.15.110:0/523770510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x5592627651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19f72640 1 -- 172.21.15.110:0/523770510 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592627686c0 con 0x5592627689b0 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.164+0000 7f3e19771640 1 --2- 172.21.15.110:0/523770510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x559262764c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:02.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.165+0000 7f3e19f72640 1 --2- 172.21.15.110:0/523770510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592627689b0 0x559262761520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c017760 tx=0x7f3e0c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39cac620f800e71e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.165+0000 7f3dfeffd640 1 -- 172.21.15.110:0/523770510 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c028070 con 0x5592627689b0 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.165+0000 7f3dfeffd640 1 -- 172.21.15.110:0/523770510 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e0c01f040 con 0x5592627689b0 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.165+0000 7f3dfeffd640 1 -- 172.21.15.110:0/523770510 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c01e040 con 0x5592627689b0 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/523770510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592627689b0 msgr2=0x559262761520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/523770510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592627689b0 0x559262761520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c017760 tx=0x7f3e0c015000 comp rx=0 tx=0).stop 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/523770510 shutdown_connections 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/523770510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592627689b0 0x559262761520 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/523770510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559262763840 0x5592627651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/523770510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x559262764c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/523770510 >> 172.21.15.110:0/523770510 conn(0x55926275e4c0 msgr2=0x55926275e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/523770510 shutdown_connections 2024-04-25T00:10:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/523770510 wait complete. 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 -- start start 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.166+0000 7f3e1e7c1740 1 Processor -- start 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x5592627711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x559262771730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e1e7c1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x559262772610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3df800c910 con 0x5592627689b0 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3df800dd80 con 0x559262763840 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e1e7c1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3df800cfe0 con 0x559262762e50 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e19771640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x5592627711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e1e7c1740 10 client.0 _ll_register_callbacks cb 0x7f3e04097540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e19771640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x5592627711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49986/0 (socket says 172.21.15.110:49986) 2024-04-25T00:10:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e19771640 1 -- 172.21.15.110:0/2596740566 learned_addr learned my addr 172.21.15.110:0/2596740566 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e18f70640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x559262771730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e19f72640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x559262772610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e19771640 1 -- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 msgr2=0x559262771730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e19771640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x559262771730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.167+0000 7f3e19771640 1 -- 172.21.15.110:0/2596740566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 msgr2=0x559262772610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.168+0000 7f3e19771640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x559262772610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.168+0000 7f3e19771640 1 -- 172.21.15.110:0/2596740566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559262772c00 con 0x559262762e50 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.168+0000 7f3e18f70640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x559262771730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.168+0000 7f3e19f72640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x559262772610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.168+0000 7f3e19771640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x5592627711f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005dc0 tx=0x7f3e10005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b0d911f037030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.168+0000 7f3dfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:10:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.168+0000 7f3dfcff9640 1 -- 172.21.15.110:0/2596740566 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10035040 con 0x559262762e50 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.168+0000 7f3dfcff9640 1 -- 172.21.15.110:0/2596740566 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3e10047040 con 0x559262762e50 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/2596740566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0c01f040 con 0x559262762e50 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3dfcff9640 1 -- 172.21.15.110:0/2596740566 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e10003a80 con 0x559262762e50 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 10 client.25453 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/2596740566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559262773f10 con 0x559262762e50 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3dfcff9640 1 -- 172.21.15.110:0/2596740566 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e0c01f040 con 0x559262762e50 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3dfcff9640 1 -- 172.21.15.110:0/2596740566 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f3e100049d0 con 0x559262762e50 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 10 client.25453 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 10 client.25453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 20 client.25453 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5592627dfaa0 0x5592627e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 4 client.25453 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:10:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.169+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/2596740566 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592626d41f0 con 0x5592627dfaa0 2024-04-25T00:10:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.170+0000 7f3e18f70640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5592627dfaa0 0x5592627e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.170+0000 7f3dfcff9640 10 client.25453 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.170+0000 7f3e18f70640 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5592627dfaa0 0x5592627e1e90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3240ba65a61888af server_cookie=7ee3416bf82cf79e in_seq=0 out_seq=0 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.189+0000 7f3dfcff9640 1 -- 172.21.15.110:0/2596740566 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592626d41f0 con 0x5592627dfaa0 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.189+0000 7f3dfcff9640 10 client.25453 handle_command_reply: tid=0 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.189+0000 7f3e1e7c1740 1 client.25453 shutdown 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.189+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/2596740566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 msgr2=0x5592627711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.189+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x5592627711f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005dc0 tx=0x7f3e10005d50 comp rx=0 tx=0).stop 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.189+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/2596740566 shutdown_connections 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.189+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5592627dfaa0 0x5592627e1e90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.189+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592627689b0 0x559262772610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.190+0000 7f3dfcff9640 0 client.25453 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.190+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559262763840 0x559262771730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.190+0000 7f3e1e7c1740 1 --2- 172.21.15.110:0/2596740566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559262762e50 0x5592627711f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.190+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/2596740566 >> 172.21.15.110:0/2596740566 conn(0x55926275e4c0 msgr2=0x559262761520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.190+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/2596740566 shutdown_connections 2024-04-25T00:10:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.190+0000 7f3e1e7c1740 1 -- 172.21.15.110:0/2596740566 wait complete. 2024-04-25T00:10:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.190+0000 7f3e1e7c1740 20 client.25453 trim_cache size 0 max 16384 2024-04-25T00:10:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:10:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:10:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:10:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:10:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e000a2cb0 msgr2=0x7f3e000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e000a2cb0 0x7f3e000a5170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x5592627640a0 tx=0x7f3e0c00d040 comp rx=0 tx=0).stop 2024-04-25T00:10:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e14101890 msgr2=0x7f3e1419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e14101890 0x7f3e1419a080 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400e9f0 tx=0x7f3e0400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/3156816350 shutdown_connections 2024-04-25T00:10:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e000a2cb0 0x7f3e000a5170 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e14102250 0x7f3e141a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e14101890 0x7f3e1419a080 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 --2- 172.21.15.110:0/3156816350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e141006a0 0x7f3e14199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/3156816350 >> 172.21.15.110:0/3156816350 conn(0x7f3e140fbc10 msgr2=0x7f3e141907c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/3156816350 shutdown_connections 2024-04-25T00:10:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:02.191+0000 7f3e1b9d5640 1 -- 172.21.15.110:0/3156816350 wait complete. 2024-04-25T00:10:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:03 smithi188 ceph-mon[33285]: pgmap v591: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.4 MiB/s wr, 1.03k op/s 2024-04-25T00:10:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:03 smithi110 ceph-mon[29187]: pgmap v591: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.4 MiB/s wr, 1.03k op/s 2024-04-25T00:10:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:03 smithi173 ceph-mon[32402]: pgmap v591: 97 pgs: 97 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.4 MiB/s wr, 1.03k op/s 2024-04-25T00:10:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:05 smithi188 ceph-mon[33285]: pgmap v592: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 952 op/s 2024-04-25T00:10:06.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:05 smithi110 ceph-mon[29187]: pgmap v592: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 952 op/s 2024-04-25T00:10:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:05 smithi173 ceph-mon[32402]: pgmap v592: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 952 op/s 2024-04-25T00:10:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 420.0 s, 822.4 tps, lat 121.925 ms stddev 21.702 2024-04-25T00:10:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:07 smithi110 ceph-mon[29187]: pgmap v593: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 879 op/s 2024-04-25T00:10:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:07 smithi173 ceph-mon[32402]: pgmap v593: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 879 op/s 2024-04-25T00:10:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:07 smithi188 ceph-mon[33285]: pgmap v593: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 879 op/s 2024-04-25T00:10:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:09 smithi110 ceph-mon[29187]: pgmap v594: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-04-25T00:10:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:09 smithi173 ceph-mon[32402]: pgmap v594: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-04-25T00:10:10.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:09 smithi188 ceph-mon[33285]: pgmap v594: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-04-25T00:10:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 425.0 s, 826.4 tps, lat 121.107 ms stddev 16.157 2024-04-25T00:10:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:11 smithi110 ceph-mon[29187]: pgmap v595: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.9 MiB/s wr, 881 op/s 2024-04-25T00:10:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:11 smithi173 ceph-mon[32402]: pgmap v595: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.9 MiB/s wr, 881 op/s 2024-04-25T00:10:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:11 smithi188 ceph-mon[33285]: pgmap v595: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.9 MiB/s wr, 881 op/s 2024-04-25T00:10:13.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:12 smithi110 ceph-mon[29187]: pgmap v596: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 989 op/s 2024-04-25T00:10:13.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:12 smithi173 ceph-mon[32402]: pgmap v596: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 989 op/s 2024-04-25T00:10:13.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:12 smithi188 ceph-mon[33285]: pgmap v596: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 989 op/s 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 -- 172.21.15.110:0/2389692456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 msgr2=0x7f0e001081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 --2- 172.21.15.110:0/2389692456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 0x7f0e001081f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009900 tx=0x7f0df402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 -- 172.21.15.110:0/2389692456 shutdown_connections 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 --2- 172.21.15.110:0/2389692456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e00109980 0x7f0e00110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 --2- 172.21.15.110:0/2389692456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e00108fc0 0x7f0e00109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 --2- 172.21.15.110:0/2389692456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 0x7f0e001081f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 -- 172.21.15.110:0/2389692456 >> 172.21.15.110:0/2389692456 conn(0x7f0e00076d90 msgr2=0x7f0e000791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 -- 172.21.15.110:0/2389692456 shutdown_connections 2024-04-25T00:10:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.326+0000 7f0e059b7640 1 -- 172.21.15.110:0/2389692456 wait complete. 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.327+0000 7f0e059b7640 1 Processor -- start 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0e059b7640 1 -- start start 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0e059b7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 0x7f0e0019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0e059b7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e00108fc0 0x7f0e0019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0e059b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e00109980 0x7f0e001a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0e059b7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001138d0 con 0x7f0e00108fc0 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0e059b7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e00113750 con 0x7f0e00107dd0 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0e059b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e00113a50 con 0x7f0e00109980 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dfeffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 0x7f0e0019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dff7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e00109980 0x7f0e001a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dfeffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 0x7f0e0019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39854/0 (socket says 172.21.15.110:39854) 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dfeffd640 1 -- 172.21.15.110:0/3402571359 learned_addr learned my addr 172.21.15.110:0/3402571359 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e00108fc0 0x7f0e0019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dff7fe640 1 -- 172.21.15.110:0/3402571359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 msgr2=0x7f0e0019df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dff7fe640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 0x7f0e0019df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dff7fe640 1 -- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e00108fc0 msgr2=0x7f0e0019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dff7fe640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e00108fc0 0x7f0e0019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dff7fe640 1 -- 172.21.15.110:0/3402571359 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e001a5b30 con 0x7f0e00109980 2024-04-25T00:10:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.328+0000 7f0dff7fe640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e00109980 0x7f0e001a54c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009fd0 tx=0x7f0df800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.329+0000 7f0e049b5640 1 -- 172.21.15.110:0/3402571359 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df800ce20 con 0x7f0e00109980 2024-04-25T00:10:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.329+0000 7f0e049b5640 1 -- 172.21.15.110:0/3402571359 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df8005020 con 0x7f0e00109980 2024-04-25T00:10:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.329+0000 7f0e049b5640 1 -- 172.21.15.110:0/3402571359 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8010690 con 0x7f0e00109980 2024-04-25T00:10:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.329+0000 7f0e059b7640 1 -- 172.21.15.110:0/3402571359 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e001a5e20 con 0x7f0e00109980 2024-04-25T00:10:15.331 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.329+0000 7f0e059b7640 1 -- 172.21.15.110:0/3402571359 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e001a6230 con 0x7f0e00109980 2024-04-25T00:10:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e049b5640 1 -- 172.21.15.110:0/3402571359 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0df80051d0 con 0x7f0e00109980 2024-04-25T00:10:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e087a1740 1 Processor -- start 2024-04-25T00:10:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e087a1740 1 -- start start 2024-04-25T00:10:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d869270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d86e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 0x55735d870e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e087a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55735d7cccb0 con 0x55735d868e50 2024-04-25T00:10:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e087a1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55735d7755b0 con 0x55735d869840 2024-04-25T00:10:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.332+0000 7f0e087a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55735d7c8c70 con 0x55735d86e9b0 2024-04-25T00:10:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d869270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfeffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d86e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfeffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d86e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39868/0 (socket says 172.21.15.110:39868) 2024-04-25T00:10:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfeffd640 1 -- 172.21.15.110:0/2465113356 learned_addr learned my addr 172.21.15.110:0/2465113356 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfeffd640 1 -- 172.21.15.110:0/2465113356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 msgr2=0x55735d870e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/2465113356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 0x55735d870e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfeffd640 1 -- 172.21.15.110:0/2465113356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 msgr2=0x55735d869270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0e049b5640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0de80a2da0 0x7f0de80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/2465113356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d869270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfeffd640 1 -- 172.21.15.110:0/2465113356 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735d4e1210 con 0x55735d869840 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/2465113356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d869270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0dff7fe640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0de80a2da0 0x7f0de80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.333+0000 7f0e049b5640 1 -- 172.21.15.110:0/3402571359 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0df80cf670 con 0x7f0e00109980 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.334+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/2465113356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d86e340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0df4005eb0 tx=0x7f0df4031af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14eaa6801f33d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.334+0000 7f0de5ffb640 1 -- 172.21.15.110:0/2465113356 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4035030 con 0x55735d869840 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.334+0000 7f0de5ffb640 1 -- 172.21.15.110:0/2465113356 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df4040070 con 0x55735d869840 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.334+0000 7f0de5ffb640 1 -- 172.21.15.110:0/2465113356 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4003930 con 0x55735d869840 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 -- 172.21.15.110:0/2465113356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 msgr2=0x55735d86e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2465113356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d86e340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0df4005eb0 tx=0x7f0df4031af0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0dff7fe640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0de80a2da0 0x7f0de80a5260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0df809bc00 tx=0x7f0df8005810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 -- 172.21.15.110:0/2465113356 shutdown_connections 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2465113356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 0x55735d870e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2465113356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d86e340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2465113356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d869270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 -- 172.21.15.110:0/2465113356 >> 172.21.15.110:0/2465113356 conn(0x55735d8644c0 msgr2=0x55735d8648c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 -- 172.21.15.110:0/2465113356 shutdown_connections 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 -- 172.21.15.110:0/2465113356 wait complete. 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 -- start start 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.335+0000 7f0e087a1740 1 Processor -- start 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d8732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d8737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d86e9b0 0x55735d87a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0e087a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55735d871900 con 0x55735d868e50 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0e087a1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55735d788b50 con 0x55735d86e9b0 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0e087a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55735d871a80 con 0x55735d869840 2024-04-25T00:10:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0e087a1740 10 client.0 _ll_register_callbacks cb 0x55735d7c3ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d8732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfeffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d8737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d8732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57800/0 (socket says 172.21.15.110:57800) 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfeffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d8737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:58996/0 (socket says 172.21.15.110:58996) 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfe7fc640 1 -- 172.21.15.110:0/2258307607 learned_addr learned my addr 172.21.15.110:0/2258307607 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d86e9b0 0x55735d87a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfeffd640 1 -- 172.21.15.110:0/2258307607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d86e9b0 msgr2=0x55735d87a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d86e9b0 0x55735d87a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfeffd640 1 -- 172.21.15.110:0/2258307607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 msgr2=0x55735d8732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d8732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.336+0000 7f0dfeffd640 1 -- 172.21.15.110:0/2258307607 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735d5ad620 con 0x55735d869840 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d8737e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009a20 tx=0x7f0df4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7efd05f62f6ea90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0ddeffd640 1 -- 172.21.15.110:0/2258307607 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4052060 con 0x55735d869840 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0e087a1740 1 -- 172.21.15.110:0/2258307607 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df4040070 con 0x55735d869840 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0ddeffd640 1 -- 172.21.15.110:0/2258307607 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0df404d480 con 0x55735d869840 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0ddeffd640 1 -- 172.21.15.110:0/2258307607 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0df4035040 con 0x55735d869840 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0e087a1740 10 client.25462 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0e087a1740 1 -- 172.21.15.110:0/2258307607 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55735d871950 con 0x55735d869840 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0ddeffd640 1 -- 172.21.15.110:0/2258307607 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df4040070 con 0x55735d869840 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.337+0000 7f0ddeffd640 1 -- 172.21.15.110:0/2258307607 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0df403c740 con 0x55735d869840 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0e087a1740 10 client.25462 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0e087a1740 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0e087a1740 20 client.25462 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55735d87ee30 0x55735d881220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0e087a1740 4 client.25462 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0e087a1740 1 -- 172.21.15.110:0/2258307607 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55735d7da1f0 con 0x55735d87ee30 2024-04-25T00:10:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55735d87ee30 0x55735d881220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55735d87ee30 0x55735d881220 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a956ed516e9408f3 server_cookie=9476b9f9ab7ed273 in_seq=0 out_seq=0 2024-04-25T00:10:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.338+0000 7f0ddeffd640 10 client.25462 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.339+0000 7f0ddeffd640 1 -- 172.21.15.110:0/2258307607 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55735d7da1f0 con 0x55735d87ee30 2024-04-25T00:10:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.339+0000 7f0ddeffd640 10 client.25462 handle_command_reply: tid=0 2024-04-25T00:10:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.339+0000 7f0e087a1740 1 client.25462 shutdown 2024-04-25T00:10:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 -- 172.21.15.110:0/2258307607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 msgr2=0x55735d8737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d8737e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009a20 tx=0x7f0df4005fd0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 -- 172.21.15.110:0/2258307607 shutdown_connections 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55735d87ee30 0x55735d881220 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d86e9b0 0x55735d87a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d8737e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 --2- 172.21.15.110:0/2258307607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d8732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0ddeffd640 0 client.25462 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 -- 172.21.15.110:0/2258307607 >> 172.21.15.110:0/2258307607 conn(0x55735d8644c0 msgr2=0x55735d8635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 -- 172.21.15.110:0/2258307607 shutdown_connections 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 1 -- 172.21.15.110:0/2258307607 wait complete. 2024-04-25T00:10:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.340+0000 7f0e087a1740 20 client.25462 trim_cache size 0 max 16384 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0e087a1740 1 Processor -- start 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0e087a1740 1 -- start start 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d867520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d87a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 0x55735d867a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0e087a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001b17d0 con 0x55735d868e50 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0e087a1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001af0a0 con 0x55735d869840 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0e087a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001acef0 con 0x55735d86e9b0 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d867520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d867520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57810/0 (socket says 172.21.15.110:57810) 2024-04-25T00:10:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0dfe7fc640 1 -- 172.21.15.110:0/3106713598 learned_addr learned my addr 172.21.15.110:0/3106713598 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.366+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 0x55735d867a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.367+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d87a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.367+0000 7f0dfe7fc640 1 -- 172.21.15.110:0/3106713598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 msgr2=0x55735d867a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.367+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 0x55735d867a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.367+0000 7f0dfe7fc640 1 -- 172.21.15.110:0/3106713598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 msgr2=0x55735d87a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.367+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d87a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.367+0000 7f0dfe7fc640 1 -- 172.21.15.110:0/3106713598 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735d864060 con 0x55735d868e50 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.367+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d867520 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0df0005350 tx=0x7f0df00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8370e276c34951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.368+0000 7f0de57fa640 1 -- 172.21.15.110:0/3106713598 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0004020 con 0x55735d868e50 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.368+0000 7f0de57fa640 1 -- 172.21.15.110:0/3106713598 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df00228f0 con 0x55735d868e50 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.368+0000 7f0de57fa640 1 -- 172.21.15.110:0/3106713598 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0003a50 con 0x55735d868e50 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 -- 172.21.15.110:0/3106713598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 msgr2=0x55735d867520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d867520 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0df0005350 tx=0x7f0df00046c0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 -- 172.21.15.110:0/3106713598 shutdown_connections 2024-04-25T00:10:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d86e9b0 0x55735d867a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d869840 0x55735d87a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 --2- 172.21.15.110:0/3106713598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d868e50 0x55735d867520 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 -- 172.21.15.110:0/3106713598 >> 172.21.15.110:0/3106713598 conn(0x55735d8644c0 msgr2=0x55735d8648a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 -- 172.21.15.110:0/3106713598 shutdown_connections 2024-04-25T00:10:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 -- 172.21.15.110:0/3106713598 wait complete. 2024-04-25T00:10:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.369+0000 7f0e087a1740 1 -- start start 2024-04-25T00:10:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0e087a1740 1 Processor -- start 2024-04-25T00:10:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d868e50 0x55735d878120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d878660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0e087a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d86e9b0 0x55735d879600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0e087a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001ad560 con 0x55735d86e9b0 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0e087a1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001ae9d0 con 0x55735d868e50 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0e087a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e001adc30 con 0x55735d869840 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0e087a1740 10 client.0 _ll_register_callbacks cb 0x7f0df4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d868e50 0x55735d878120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0dfe7fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d868e50 0x55735d878120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:39904/0 (socket says 172.21.15.110:39904) 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0dfe7fc640 1 -- 172.21.15.110:0/942699666 learned_addr learned my addr 172.21.15.110:0/942699666 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d86e9b0 0x55735d879600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d878660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0dfeffd640 1 -- 172.21.15.110:0/942699666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d868e50 msgr2=0x55735d878120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d868e50 0x55735d878120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.370+0000 7f0dfeffd640 1 -- 172.21.15.110:0/942699666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d86e9b0 msgr2=0x55735d879600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d86e9b0 0x55735d879600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0dfeffd640 1 -- 172.21.15.110:0/942699666 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735d879bf0 con 0x55735d869840 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d86e9b0 0x55735d879600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0dfeffd640 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d878660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0df403cb60 tx=0x7f0df4031a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d151423f1666ca68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0ddeffd640 1 -- 172.21.15.110:0/942699666 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df405d070 con 0x55735d869840 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0ddeffd640 1 -- 172.21.15.110:0/942699666 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0df403c850 con 0x55735d869840 2024-04-25T00:10:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0e087a1740 1 -- 172.21.15.110:0/942699666 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df00228f0 con 0x55735d869840 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0ddeffd640 1 -- 172.21.15.110:0/942699666 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0df4052040 con 0x55735d869840 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0e087a1740 10 client.25468 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.371+0000 7f0e087a1740 1 -- 172.21.15.110:0/942699666 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55735d8e54a0 con 0x55735d869840 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0ddeffd640 1 -- 172.21.15.110:0/942699666 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df00228f0 con 0x55735d869840 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0ddeffd640 1 -- 172.21.15.110:0/942699666 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0df4004ae0 con 0x55735d869840 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0e087a1740 10 client.25468 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0e087a1740 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0e087a1740 20 client.25468 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0e087a1740 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55735d8e5e70 0x55735d8e8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0e087a1740 4 client.25468 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0e087a1740 1 -- 172.21.15.110:0/942699666 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55735d7d4540 con 0x55735d8e5e70 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.372+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55735d8e5e70 0x55735d8e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.373+0000 7f0dfe7fc640 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55735d8e5e70 0x55735d8e8290 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30eb5b05139c93ec server_cookie=21a2b92b7dcb6ee0 in_seq=0 out_seq=0 2024-04-25T00:10:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.373+0000 7f0ddeffd640 10 client.25468 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0ddeffd640 1 -- 172.21.15.110:0/942699666 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55735d7d4540 con 0x55735d8e5e70 2024-04-25T00:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0ddeffd640 10 client.25468 handle_command_reply: tid=0 2024-04-25T00:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 client.25468 shutdown 2024-04-25T00:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 -- 172.21.15.110:0/942699666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 msgr2=0x55735d878660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d878660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0df403cb60 tx=0x7f0df4031a40 comp rx=0 tx=0).stop 2024-04-25T00:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 -- 172.21.15.110:0/942699666 shutdown_connections 2024-04-25T00:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55735d8e5e70 0x55735d8e8290 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55735d86e9b0 0x55735d879600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55735d869840 0x55735d878660 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 --2- 172.21.15.110:0/942699666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55735d868e50 0x55735d878120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 -- 172.21.15.110:0/942699666 >> 172.21.15.110:0/942699666 conn(0x55735d8644c0 msgr2=0x55735d867520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0ddeffd640 0 client.25468 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 -- 172.21.15.110:0/942699666 shutdown_connections 2024-04-25T00:10:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 1 -- 172.21.15.110:0/942699666 wait complete. 2024-04-25T00:10:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.393+0000 7f0e087a1740 20 client.25468 trim_cache size 0 max 16384 2024-04-25T00:10:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:10:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:10:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:10:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 -- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0de80a2da0 msgr2=0x7f0de80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0de80a2da0 0x7f0de80a5260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0df809bc00 tx=0x7f0df8005810 comp rx=0 tx=0).stop 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 -- 172.21.15.110:0/3402571359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e00109980 msgr2=0x7f0e001a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e00109980 0x7f0e001a54c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009fd0 tx=0x7f0df800ef10 comp rx=0 tx=0).stop 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 -- 172.21.15.110:0/3402571359 shutdown_connections 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0de80a2da0 0x7f0de80a5260 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e00109980 0x7f0e001a54c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e00108fc0 0x7f0e0019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 --2- 172.21.15.110:0/3402571359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e00107dd0 0x7f0e0019df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.394+0000 7f0e059b7640 1 -- 172.21.15.110:0/3402571359 >> 172.21.15.110:0/3402571359 conn(0x7f0e00076d90 msgr2=0x7f0e0010c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.395+0000 7f0e059b7640 1 -- 172.21.15.110:0/3402571359 shutdown_connections 2024-04-25T00:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:15.395+0000 7f0e059b7640 1 -- 172.21.15.110:0/3402571359 wait complete. 2024-04-25T00:10:16.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:15 smithi188 ceph-mon[33285]: pgmap v597: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 1.05k op/s 2024-04-25T00:10:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:15 smithi110 ceph-mon[29187]: pgmap v597: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 1.05k op/s 2024-04-25T00:10:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:15 smithi173 ceph-mon[32402]: pgmap v597: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 1.05k op/s 2024-04-25T00:10:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 430.0 s, 833.4 tps, lat 119.888 ms stddev 16.716 2024-04-25T00:10:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:17 smithi188 ceph-mon[33285]: pgmap v598: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 876 op/s 2024-04-25T00:10:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:17 smithi110 ceph-mon[29187]: pgmap v598: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 876 op/s 2024-04-25T00:10:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:17 smithi173 ceph-mon[32402]: pgmap v598: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 876 op/s 2024-04-25T00:10:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:19 smithi188 ceph-mon[33285]: pgmap v599: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 999 op/s 2024-04-25T00:10:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:19 smithi110 ceph-mon[29187]: pgmap v599: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 999 op/s 2024-04-25T00:10:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:19 smithi173 ceph-mon[32402]: pgmap v599: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 999 op/s 2024-04-25T00:10:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 435.0 s, 795.6 tps, lat 125.895 ms stddev 23.135 2024-04-25T00:10:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:21 smithi188 ceph-mon[33285]: pgmap v600: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 845 op/s 2024-04-25T00:10:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:21 smithi110 ceph-mon[29187]: pgmap v600: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 845 op/s 2024-04-25T00:10:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:21 smithi173 ceph-mon[32402]: pgmap v600: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.6 MiB/s wr, 845 op/s 2024-04-25T00:10:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:23 smithi188 ceph-mon[33285]: pgmap v601: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.3 MiB/s wr, 935 op/s 2024-04-25T00:10:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:23 smithi110 ceph-mon[29187]: pgmap v601: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.3 MiB/s wr, 935 op/s 2024-04-25T00:10:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:23 smithi173 ceph-mon[32402]: pgmap v601: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.3 MiB/s wr, 935 op/s 2024-04-25T00:10:26.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:25 smithi188 ceph-mon[33285]: pgmap v602: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 1.03k op/s 2024-04-25T00:10:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:25 smithi110 ceph-mon[29187]: pgmap v602: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 1.03k op/s 2024-04-25T00:10:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:25 smithi173 ceph-mon[32402]: pgmap v602: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.3 MiB/s wr, 1.03k op/s 2024-04-25T00:10:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 440.0 s, 788.2 tps, lat 126.827 ms stddev 41.388 2024-04-25T00:10:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:27 smithi188 ceph-mon[33285]: pgmap v603: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 788 op/s 2024-04-25T00:10:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:27 smithi110 ceph-mon[29187]: pgmap v603: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 788 op/s 2024-04-25T00:10:28.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:27 smithi173 ceph-mon[32402]: pgmap v603: 97 pgs: 97 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 788 op/s 2024-04-25T00:10:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 -- 172.21.15.110:0/1200772648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10d3a0 msgr2=0x7f798c10d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 --2- 172.21.15.110:0/1200772648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10d3a0 0x7f798c10d820 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f797c009900 tx=0x7f797c02a410 comp rx=0 tx=0).stop 2024-04-25T00:10:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 -- 172.21.15.110:0/1200772648 shutdown_connections 2024-04-25T00:10:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 --2- 172.21.15.110:0/1200772648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f798c10dd60 0x7f798c1145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 --2- 172.21.15.110:0/1200772648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10d3a0 0x7f798c10d820 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 --2- 172.21.15.110:0/1200772648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f798c10c1b0 0x7f798c10c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 -- 172.21.15.110:0/1200772648 >> 172.21.15.110:0/1200772648 conn(0x7f798c06c380 msgr2=0x7f798c06c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 -- 172.21.15.110:0/1200772648 shutdown_connections 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.516+0000 7f7993e72640 1 -- 172.21.15.110:0/1200772648 wait complete. 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7993e72640 1 Processor -- start 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7993e72640 1 -- start start 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7993e72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f798c10c1b0 0x7f798c1a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7993e72640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10dd60 0x7f798c1a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7993e72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f798c1a7aa0 0x7f798c1a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7993e72640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f798c117d40 con 0x7f798c10dd60 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7993e72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f798c117bc0 con 0x7f798c10c1b0 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7993e72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f798c117ec0 con 0x7f798c1a7aa0 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7991c0e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f798c10c1b0 0x7f798c1a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f799140d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10dd60 0x7f798c1a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f7991c0e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f798c10c1b0 0x7f798c1a2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47940/0 (socket says 172.21.15.110:47940) 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f799140d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10dd60 0x7f798c1a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36126/0 (socket says 172.21.15.110:36126) 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.517+0000 7f799140d640 1 -- 172.21.15.110:0/4287555716 learned_addr learned my addr 172.21.15.110:0/4287555716 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f799240f640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f798c1a7aa0 0x7f798c1a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7991c0e640 1 -- 172.21.15.110:0/4287555716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f798c1a7aa0 msgr2=0x7f798c1a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7991c0e640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f798c1a7aa0 0x7f798c1a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7991c0e640 1 -- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10dd60 msgr2=0x7f798c1a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7991c0e640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10dd60 0x7f798c1a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7991c0e640 1 -- 172.21.15.110:0/4287555716 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c1aa500 con 0x7f798c10c1b0 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f799240f640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f798c1a7aa0 0x7f798c1a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f799140d640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10dd60 0x7f798c1a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7991c0e640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f798c10c1b0 0x7f798c1a2500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7984005fd0 tx=0x7f798400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7982ffd640 1 -- 172.21.15.110:0/4287555716 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984003970 con 0x7f798c10c1b0 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7993e72640 1 -- 172.21.15.110:0/4287555716 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c1aa7f0 con 0x7f798c10c1b0 2024-04-25T00:10:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7993e72640 1 -- 172.21.15.110:0/4287555716 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c1aad60 con 0x7f798c10c1b0 2024-04-25T00:10:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7982ffd640 1 -- 172.21.15.110:0/4287555716 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7984004030 con 0x7f798c10c1b0 2024-04-25T00:10:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.518+0000 7f7982ffd640 1 -- 172.21.15.110:0/4287555716 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984010720 con 0x7f798c10c1b0 2024-04-25T00:10:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.521+0000 7f7982ffd640 1 -- 172.21.15.110:0/4287555716 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f79840041e0 con 0x7f798c10c1b0 2024-04-25T00:10:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7996b80740 1 Processor -- start 2024-04-25T00:10:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7996b80740 1 -- start start 2024-04-25T00:10:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7982ffd640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79780a2cb0 0x7f79780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7996b80740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ebce50 0x5651d4ebd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7996b80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ec2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7996b80740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ec29b0 0x5651d4ec4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7996b80740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5651d4e20cb0 con 0x5651d4ec29b0 2024-04-25T00:10:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7996b80740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5651d4dc95b0 con 0x5651d4ebce50 2024-04-25T00:10:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.522+0000 7f7996b80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5651d4e1cc70 con 0x5651d4ebd840 2024-04-25T00:10:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7982ffd640 1 -- 172.21.15.110:0/4287555716 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f79840cf4c0 con 0x7f798c10c1b0 2024-04-25T00:10:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7991c0e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ebce50 0x5651d4ebd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7991c0e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ebce50 0x5651d4ebd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47944/0 (socket says 172.21.15.110:47944) 2024-04-25T00:10:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7991c0e640 1 -- 172.21.15.110:0/1045104063 learned_addr learned my addr 172.21.15.110:0/1045104063 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f799240f640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79780a2cb0 0x7f79780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f799140d640 1 --2- 172.21.15.110:0/1045104063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ec2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7991c0e640 1 -- 172.21.15.110:0/1045104063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 msgr2=0x5651d4ec2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7991c0e640 1 --2- 172.21.15.110:0/1045104063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ec2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7991c0e640 1 -- 172.21.15.110:0/1045104063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ec29b0 msgr2=0x5651d4ec4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7991c0e640 1 --2- 172.21.15.110:0/1045104063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ec29b0 0x5651d4ec4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.523+0000 7f7991c0e640 1 -- 172.21.15.110:0/1045104063 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d4b35210 con 0x5651d4ebce50 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.524+0000 7f7991c0e640 1 --2- 172.21.15.110:0/1045104063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ebce50 0x5651d4ebd270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79840981b0 tx=0x7f798409b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38c6b2d416f2ead1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.524+0000 7f799240f640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79780a2cb0 0x7f79780a5170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7988009cc0 tx=0x7f7988009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.524+0000 7f7977fff640 1 -- 172.21.15.110:0/1045104063 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798401d1c0 con 0x5651d4ebce50 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.524+0000 7f7977fff640 1 -- 172.21.15.110:0/1045104063 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7984007880 con 0x5651d4ebce50 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.524+0000 7f7977fff640 1 -- 172.21.15.110:0/1045104063 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79840983c0 con 0x5651d4ebce50 2024-04-25T00:10:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 -- 172.21.15.110:0/1045104063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ebce50 msgr2=0x5651d4ebd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 --2- 172.21.15.110:0/1045104063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ebce50 0x5651d4ebd270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79840981b0 tx=0x7f798409b6b0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 -- 172.21.15.110:0/1045104063 shutdown_connections 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 --2- 172.21.15.110:0/1045104063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ec29b0 0x5651d4ec4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 --2- 172.21.15.110:0/1045104063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ec2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 --2- 172.21.15.110:0/1045104063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ebce50 0x5651d4ebd270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 -- 172.21.15.110:0/1045104063 >> 172.21.15.110:0/1045104063 conn(0x5651d4eb84c0 msgr2=0x5651d4eb88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 -- 172.21.15.110:0/1045104063 shutdown_connections 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 -- 172.21.15.110:0/1045104063 wait complete. 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.525+0000 7f7996b80740 1 -- start start 2024-04-25T00:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7996b80740 1 Processor -- start 2024-04-25T00:10:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7996b80740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ec72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7996b80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ec77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7996b80740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ece830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7996b80740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5651d4ec5960 con 0x5651d4ebce50 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7996b80740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5651d4ddcb50 con 0x5651d4ec29b0 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7991c0e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ec72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7991c0e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ec72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36156/0 (socket says 172.21.15.110:36156) 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7991c0e640 1 -- 172.21.15.110:0/3590547125 learned_addr learned my addr 172.21.15.110:0/3590547125 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.526+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5651d4ec5ae0 con 0x5651d4ebd840 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f799240f640 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ece830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f799140d640 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ec77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f7996b80740 10 client.0 _ll_register_callbacks cb 0x5651d4e17ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f799240f640 1 -- 172.21.15.110:0/3590547125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 msgr2=0x5651d4ec77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f799240f640 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ec77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f799240f640 1 -- 172.21.15.110:0/3590547125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 msgr2=0x5651d4ec72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f799240f640 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ec72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f799240f640 1 -- 172.21.15.110:0/3590547125 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d4c01620 con 0x5651d4ec29b0 2024-04-25T00:10:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f799240f640 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ece830 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7988013c60 tx=0x7f7988004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9acdf6f724ec7461 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f79757fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f79757fa640 1 -- 172.21.15.110:0/3590547125 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798800ec60 con 0x5651d4ec29b0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7984007880 con 0x5651d4ec29b0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.527+0000 7f79757fa640 1 -- 172.21.15.110:0/3590547125 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f798800ee00 con 0x5651d4ec29b0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f79757fa640 1 -- 172.21.15.110:0/3590547125 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7988026400 con 0x5651d4ec29b0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f7996b80740 10 client.25511 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5651d4ed1bd0 con 0x5651d4ec29b0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f79757fa640 1 -- 172.21.15.110:0/3590547125 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7988026a50 con 0x5651d4ec29b0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f79757fa640 1 -- 172.21.15.110:0/3590547125 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7988031770 con 0x5651d4ec29b0 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f7996b80740 10 client.25511 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f7996b80740 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.528+0000 7f7996b80740 20 client.25511 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.529+0000 7f7996b80740 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651d4ed3310 0x5651d4ed5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.529+0000 7f7996b80740 4 client.25511 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.529+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7984007880 con 0x5651d4ed3310 2024-04-25T00:10:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.529+0000 7f7991c0e640 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651d4ed3310 0x5651d4ed5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.529+0000 7f7991c0e640 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651d4ed3310 0x5651d4ed5720 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f199e157953aec server_cookie=bf92aba4ee3dde64 in_seq=0 out_seq=0 2024-04-25T00:10:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.529+0000 7f79757fa640 10 client.25511 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.530+0000 7f79757fa640 1 -- 172.21.15.110:0/3590547125 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f7984007880 con 0x5651d4ed3310 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.530+0000 7f79757fa640 10 client.25511 handle_command_reply: tid=0 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.530+0000 7f7996b80740 1 client.25511 shutdown 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.530+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 msgr2=0x5651d4ece830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.530+0000 7f7996b80740 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ece830 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7988013c60 tx=0x7f7988004730 comp rx=0 tx=0).stop 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 shutdown_connections 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651d4ed3310 0x5651d4ed5720 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ece830 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ec77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 1 --2- 172.21.15.110:0/3590547125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ec72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 >> 172.21.15.110:0/3590547125 conn(0x5651d4eb84c0 msgr2=0x5651d4eb75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f79757fa640 0 client.25511 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 shutdown_connections 2024-04-25T00:10:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 1 -- 172.21.15.110:0/3590547125 wait complete. 2024-04-25T00:10:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.531+0000 7f7996b80740 20 client.25511 trim_cache size 0 max 16384 2024-04-25T00:10:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.554+0000 7f7996b80740 1 Processor -- start 2024-04-25T00:10:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.554+0000 7f7996b80740 1 -- start start 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7996b80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebce50 0x5651d4eb6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7996b80740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebd840 0x5651d4ece750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7996b80740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4eb7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7996b80740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b62c0 con 0x5651d4ebd840 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7996b80740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b3b90 con 0x5651d4ec29b0 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7996b80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b19e0 con 0x5651d4ebce50 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f799140d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebd840 0x5651d4ece750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f799140d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebd840 0x5651d4ece750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36172/0 (socket says 172.21.15.110:36172) 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f799140d640 1 -- 172.21.15.110:0/2839657865 learned_addr learned my addr 172.21.15.110:0/2839657865 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7991c0e640 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebce50 0x5651d4eb6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f799240f640 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4eb7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7991c0e640 1 -- 172.21.15.110:0/2839657865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 msgr2=0x5651d4eb7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7991c0e640 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4eb7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.555+0000 7f7991c0e640 1 -- 172.21.15.110:0/2839657865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebd840 msgr2=0x5651d4ece750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.556+0000 7f7991c0e640 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebd840 0x5651d4ece750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.556+0000 7f7991c0e640 1 -- 172.21.15.110:0/2839657865 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d4eb2850 con 0x5651d4ebce50 2024-04-25T00:10:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.556+0000 7f799140d640 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebd840 0x5651d4ece750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.556+0000 7f7991c0e640 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebce50 0x5651d4eb6d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79840a4490 tx=0x7f798409b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c12b64678aef530d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.556+0000 7f79777fe640 1 -- 172.21.15.110:0/2839657865 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798407e070 con 0x5651d4ebce50 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.556+0000 7f79777fe640 1 -- 172.21.15.110:0/2839657865 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79840a0100 con 0x5651d4ebce50 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.556+0000 7f79777fe640 1 -- 172.21.15.110:0/2839657865 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798407abc0 con 0x5651d4ebce50 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 -- 172.21.15.110:0/2839657865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebce50 msgr2=0x5651d4eb6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebce50 0x5651d4eb6d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79840a4490 tx=0x7f798409b7e0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 -- 172.21.15.110:0/2839657865 shutdown_connections 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4eb7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebd840 0x5651d4ece750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 --2- 172.21.15.110:0/2839657865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebce50 0x5651d4eb6d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 -- 172.21.15.110:0/2839657865 >> 172.21.15.110:0/2839657865 conn(0x5651d4eb84c0 msgr2=0x5651d4ebed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 -- 172.21.15.110:0/2839657865 shutdown_connections 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.557+0000 7f7996b80740 1 -- 172.21.15.110:0/2839657865 wait complete. 2024-04-25T00:10:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.558+0000 7f7996b80740 1 -- start start 2024-04-25T00:10:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.558+0000 7f7996b80740 1 Processor -- start 2024-04-25T00:10:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7996b80740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ecbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7996b80740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ecc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7996b80740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ecd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7996b80740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b2050 con 0x5651d4ebce50 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7996b80740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b34c0 con 0x5651d4ec29b0 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7996b80740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f798c1b2720 con 0x5651d4ebd840 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7996b80740 10 client.0 _ll_register_callbacks cb 0x7f7988009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799140d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ecc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799140d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ecc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:54254/0 (socket says 172.21.15.110:54254) 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799140d640 1 -- 172.21.15.110:0/2114334843 learned_addr learned my addr 172.21.15.110:0/2114334843 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799240f640 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ecd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7991c0e640 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ecbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799140d640 1 -- 172.21.15.110:0/2114334843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 msgr2=0x5651d4ecd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799140d640 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ecd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799140d640 1 -- 172.21.15.110:0/2114334843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 msgr2=0x5651d4ecbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799140d640 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ecbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f799140d640 1 -- 172.21.15.110:0/2114334843 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d4ecd6a0 con 0x5651d4ebd840 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.559+0000 7f7991c0e640 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ecbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.560+0000 7f799140d640 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ecc0f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f797c005ab0 tx=0x7f797c003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b1c7c6329e39f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.560+0000 7f79757fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.560+0000 7f79757fa640 1 -- 172.21.15.110:0/2114334843 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c04b060 con 0x5651d4ebd840 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.560+0000 7f79757fa640 1 -- 172.21.15.110:0/2114334843 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f797c003dc0 con 0x5651d4ebd840 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.560+0000 7f7996b80740 1 -- 172.21.15.110:0/2114334843 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79840a0100 con 0x5651d4ebd840 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.560+0000 7f79757fa640 1 -- 172.21.15.110:0/2114334843 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f797c035050 con 0x5651d4ebd840 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.560+0000 7f7996b80740 10 client.25486 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.560+0000 7f7996b80740 1 -- 172.21.15.110:0/2114334843 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5651d4ecea50 con 0x5651d4ebd840 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.561+0000 7f79757fa640 1 -- 172.21.15.110:0/2114334843 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79840a0100 con 0x5651d4ebd840 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.561+0000 7f79757fa640 1 -- 172.21.15.110:0/2114334843 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f797c045070 con 0x5651d4ebd840 2024-04-25T00:10:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.561+0000 7f7996b80740 10 client.25486 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.561+0000 7f7996b80740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.561+0000 7f7996b80740 20 client.25486 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.561+0000 7f7996b80740 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651d4f39fb0 0x5651d4f3c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.561+0000 7f7996b80740 4 client.25486 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:10:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.561+0000 7f7996b80740 1 -- 172.21.15.110:0/2114334843 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651d4e2e1f0 con 0x5651d4f39fb0 2024-04-25T00:10:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.562+0000 7f7991c0e640 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651d4f39fb0 0x5651d4f3c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.562+0000 7f79757fa640 10 client.25486 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.562+0000 7f7991c0e640 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651d4f39fb0 0x5651d4f3c3d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c6422d0d2cdd56 server_cookie=469a4ece5eef8090 in_seq=0 out_seq=0 2024-04-25T00:10:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.583+0000 7f79757fa640 1 -- 172.21.15.110:0/2114334843 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651d4e2e1f0 con 0x5651d4f39fb0 2024-04-25T00:10:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.583+0000 7f79757fa640 10 client.25486 handle_command_reply: tid=0 2024-04-25T00:10:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.584+0000 7f7996b80740 1 client.25486 shutdown 2024-04-25T00:10:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.584+0000 7f7996b80740 1 -- 172.21.15.110:0/2114334843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 msgr2=0x5651d4ecc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.584+0000 7f7996b80740 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ecc0f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f797c005ab0 tx=0x7f797c003890 comp rx=0 tx=0).stop 2024-04-25T00:10:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.584+0000 7f7996b80740 1 -- 172.21.15.110:0/2114334843 shutdown_connections 2024-04-25T00:10:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.585+0000 7f7996b80740 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5651d4f39fb0 0x5651d4f3c3d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.585+0000 7f7996b80740 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d4ec29b0 0x5651d4ecd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.585+0000 7f7996b80740 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651d4ebd840 0x5651d4ecc0f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.585+0000 7f7996b80740 1 --2- 172.21.15.110:0/2114334843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651d4ebce50 0x5651d4ecbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.585+0000 7f79757fa640 0 client.25486 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.585+0000 7f7996b80740 1 -- 172.21.15.110:0/2114334843 >> 172.21.15.110:0/2114334843 conn(0x5651d4eb84c0 msgr2=0x5651d4ebb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.585+0000 7f7996b80740 1 -- 172.21.15.110:0/2114334843 shutdown_connections 2024-04-25T00:10:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.585+0000 7f7996b80740 1 -- 172.21.15.110:0/2114334843 wait complete. 2024-04-25T00:10:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.586+0000 7f7996b80740 20 client.25486 trim_cache size 0 max 16384 2024-04-25T00:10:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:10:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:10:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:10:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:10:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 -- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79780a2cb0 msgr2=0x7f79780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79780a2cb0 0x7f79780a5170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7988009cc0 tx=0x7f7988009800 comp rx=0 tx=0).stop 2024-04-25T00:10:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 -- 172.21.15.110:0/4287555716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f798c10c1b0 msgr2=0x7f798c1a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f798c10c1b0 0x7f798c1a2500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7984005fd0 tx=0x7f798400ef90 comp rx=0 tx=0).stop 2024-04-25T00:10:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 -- 172.21.15.110:0/4287555716 shutdown_connections 2024-04-25T00:10:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f79780a2cb0 0x7f79780a5170 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f798c1a7aa0 0x7f798c1a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f798c10dd60 0x7f798c1a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 --2- 172.21.15.110:0/4287555716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f798c10c1b0 0x7f798c1a2500 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.587+0000 7f7993e72640 1 -- 172.21.15.110:0/4287555716 >> 172.21.15.110:0/4287555716 conn(0x7f798c06c380 msgr2=0x7f798c1106f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.588+0000 7f7993e72640 1 -- 172.21.15.110:0/4287555716 shutdown_connections 2024-04-25T00:10:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:28.588+0000 7f7993e72640 1 -- 172.21.15.110:0/4287555716 wait complete. 2024-04-25T00:10:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:29 smithi188 ceph-mon[33285]: pgmap v604: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-04-25T00:10:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:29 smithi110 ceph-mon[29187]: pgmap v604: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-04-25T00:10:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:29 smithi173 ceph-mon[32402]: pgmap v604: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-04-25T00:10:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 445.0 s, 830.2 tps, lat 120.366 ms stddev 11.926 2024-04-25T00:10:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:31 smithi188 ceph-mon[33285]: pgmap v605: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 1.01k op/s 2024-04-25T00:10:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:31 smithi110 ceph-mon[29187]: pgmap v605: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 1.01k op/s 2024-04-25T00:10:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:31 smithi173 ceph-mon[32402]: pgmap v605: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 1.01k op/s 2024-04-25T00:10:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:33 smithi188 ceph-mon[33285]: pgmap v606: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.1 MiB/s wr, 1.05k op/s 2024-04-25T00:10:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:33 smithi110 ceph-mon[29187]: pgmap v606: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.1 MiB/s wr, 1.05k op/s 2024-04-25T00:10:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:33 smithi173 ceph-mon[32402]: pgmap v606: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.1 MiB/s wr, 1.05k op/s 2024-04-25T00:10:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:35 smithi188 ceph-mon[33285]: pgmap v607: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-04-25T00:10:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:35 smithi110 ceph-mon[29187]: pgmap v607: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-04-25T00:10:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:35 smithi173 ceph-mon[32402]: pgmap v607: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-04-25T00:10:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 450.0 s, 821.4 tps, lat 121.802 ms stddev 13.322 2024-04-25T00:10:37.003 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:36 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:36 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:37.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:36 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:38.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:37 smithi188 ceph-mon[33285]: pgmap v608: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-04-25T00:10:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:37 smithi110 ceph-mon[29187]: pgmap v608: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-04-25T00:10:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:37 smithi173 ceph-mon[32402]: pgmap v608: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-04-25T00:10:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:10:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:10:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:10:40.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:39 smithi188 ceph-mon[33285]: pgmap v609: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-04-25T00:10:40.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:39 smithi110 ceph-mon[29187]: pgmap v609: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-04-25T00:10:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:39 smithi173 ceph-mon[32402]: pgmap v609: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.7 MiB/s wr, 1.13k op/s 2024-04-25T00:10:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 455.0 s, 827.2 tps, lat 120.800 ms stddev 17.955 2024-04-25T00:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 -- 172.21.15.110:0/1192736869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4108fc0 msgr2=0x7f68b4109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 --2- 172.21.15.110:0/1192736869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4108fc0 0x7f68b4109440 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009900 tx=0x7f68a002a410 comp rx=0 tx=0).stop 2024-04-25T00:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 -- 172.21.15.110:0/1192736869 shutdown_connections 2024-04-25T00:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 --2- 172.21.15.110:0/1192736869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b4109980 0x7f68b4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 --2- 172.21.15.110:0/1192736869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4108fc0 0x7f68b4109440 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 --2- 172.21.15.110:0/1192736869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b4107dd0 0x7f68b41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 -- 172.21.15.110:0/1192736869 >> 172.21.15.110:0/1192736869 conn(0x7f68b4076d90 msgr2=0x7f68b40791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 -- 172.21.15.110:0/1192736869 shutdown_connections 2024-04-25T00:10:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 -- 172.21.15.110:0/1192736869 wait complete. 2024-04-25T00:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.943+0000 7f68baa66640 1 Processor -- start 2024-04-25T00:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68baa66640 1 -- start start 2024-04-25T00:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68baa66640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b4107dd0 0x7f68b419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68baa66640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b4108fc0 0x7f68b419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68baa66640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4109980 0x7f68b41a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68baa66640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4113930 con 0x7f68b4109980 2024-04-25T00:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68baa66640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41137b0 con 0x7f68b4107dd0 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68baa66640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4113ab0 con 0x7f68b4108fc0 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4109980 0x7f68b41a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b8802640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b4107dd0 0x7f68b419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4109980 0x7f68b41a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37990/0 (socket says 172.21.15.110:37990) 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b4108fc0 0x7f68b419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 -- 172.21.15.110:0/1467653678 learned_addr learned my addr 172.21.15.110:0/1467653678 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 -- 172.21.15.110:0/1467653678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b4108fc0 msgr2=0x7f68b419e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b4108fc0 0x7f68b419e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 -- 172.21.15.110:0/1467653678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b4107dd0 msgr2=0x7f68b419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b4107dd0 0x7f68b419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 -- 172.21.15.110:0/1467653678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b41a5b70 con 0x7f68b4109980 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.944+0000 7f68b9003640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4109980 0x7f68b41a5500 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00eb10 tx=0x7f68ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.945+0000 7f68b1ffb640 1 -- 172.21.15.110:0/1467653678 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac004b20 con 0x7f68b4109980 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.945+0000 7f68baa66640 1 -- 172.21.15.110:0/1467653678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b41a5e60 con 0x7f68b4109980 2024-04-25T00:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.945+0000 7f68b1ffb640 1 -- 172.21.15.110:0/1467653678 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac004cc0 con 0x7f68b4109980 2024-04-25T00:10:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.945+0000 7f68b1ffb640 1 -- 172.21.15.110:0/1467653678 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac018630 con 0x7f68b4109980 2024-04-25T00:10:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.945+0000 7f68baa66640 1 -- 172.21.15.110:0/1467653678 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b41a62d0 con 0x7f68b4109980 2024-04-25T00:10:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.949+0000 7f68b1ffb640 1 -- 172.21.15.110:0/1467653678 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f68ac005000 con 0x7f68b4109980 2024-04-25T00:10:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.949+0000 7f68bd770740 1 Processor -- start 2024-04-25T00:10:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.949+0000 7f68bd770740 1 -- start start 2024-04-25T00:10:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68bd770740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a026270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68bd770740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a02b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68bd770740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 0x55eb7a02de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68bd770740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eb79f89cb0 con 0x55eb7a02b9b0 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68bd770740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eb79f325b0 con 0x55eb7a025e50 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68bd770740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb79f85c70 con 0x55eb7a026840 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68b1ffb640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f68a40a2cb0 0x7f68a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68b3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a026270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68b3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a026270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:44844/0 (socket says 172.21.15.110:44844) 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68b3fff640 1 -- 172.21.15.110:0/2929141982 learned_addr learned my addr 172.21.15.110:0/2929141982 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68b3fff640 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 0x55eb7a02de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68b8802640 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a02b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68b1ffb640 1 -- 172.21.15.110:0/1467653678 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f68ac0cf790 con 0x7f68b4109980 2024-04-25T00:10:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.950+0000 7f68b9003640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f68a40a2cb0 0x7f68a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f68b3fff640 1 -- 172.21.15.110:0/2929141982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 msgr2=0x55eb7a02b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f68b3fff640 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a02b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f68b3fff640 1 -- 172.21.15.110:0/2929141982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 msgr2=0x55eb7a02de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f68b3fff640 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 0x55eb7a02de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f68b3fff640 1 -- 172.21.15.110:0/2929141982 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb79c9e210 con 0x55eb7a025e50 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f68b8802640 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a02b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f68b9003640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f68a40a2cb0 0x7f68a40a5170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68ac098990 tx=0x7f68ac0bc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f68b3fff640 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a026270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f68a0006010 tx=0x7f68a0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f8b4877447357c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f689effd640 1 -- 172.21.15.110:0/2929141982 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0035040 con 0x55eb7a025e50 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f689effd640 1 -- 172.21.15.110:0/2929141982 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a0003aa0 con 0x55eb7a025e50 2024-04-25T00:10:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.951+0000 7f689effd640 1 -- 172.21.15.110:0/2929141982 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0031a80 con 0x55eb7a025e50 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.952+0000 7f68bd770740 1 -- 172.21.15.110:0/2929141982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 msgr2=0x55eb7a026270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.952+0000 7f68bd770740 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a026270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f68a0006010 tx=0x7f68a0005d50 comp rx=0 tx=0).stop 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.952+0000 7f68bd770740 1 -- 172.21.15.110:0/2929141982 shutdown_connections 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.952+0000 7f68bd770740 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 0x55eb7a02de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.952+0000 7f68bd770740 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a02b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.952+0000 7f68bd770740 1 --2- 172.21.15.110:0/2929141982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a026270 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.952+0000 7f68bd770740 1 -- 172.21.15.110:0/2929141982 >> 172.21.15.110:0/2929141982 conn(0x55eb7a0214c0 msgr2=0x55eb7a0218c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.953+0000 7f68bd770740 1 -- 172.21.15.110:0/2929141982 shutdown_connections 2024-04-25T00:10:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.953+0000 7f68bd770740 1 -- 172.21.15.110:0/2929141982 wait complete. 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.953+0000 7f68bd770740 1 -- start start 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.953+0000 7f68bd770740 1 Processor -- start 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.954+0000 7f68bd770740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a0302d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.954+0000 7f68bd770740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a030810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.954+0000 7f68bd770740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a02b9b0 0x55eb7a037850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.954+0000 7f68bd770740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eb7a02e900 con 0x55eb7a025e50 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.954+0000 7f68bd770740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eb79f45b50 con 0x55eb7a02b9b0 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.954+0000 7f68b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a0302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.954+0000 7f68bd770740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb7a02ea80 con 0x55eb7a026840 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a0302d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38006/0 (socket says 172.21.15.110:38006) 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b3fff640 1 -- 172.21.15.110:0/3811506382 learned_addr learned my addr 172.21.15.110:0/3811506382 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b8802640 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a030810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68bd770740 10 client.0 _ll_register_callbacks cb 0x55eb79f80ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b3fff640 1 -- 172.21.15.110:0/3811506382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 msgr2=0x55eb7a030810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b3fff640 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a030810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b3fff640 1 -- 172.21.15.110:0/3811506382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a02b9b0 msgr2=0x55eb7a037850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:10:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b3fff640 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a02b9b0 0x55eb7a037850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b3fff640 1 -- 172.21.15.110:0/3811506382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb79d6a620 con 0x55eb7a025e50 2024-04-25T00:10:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f68b3fff640 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a0302d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f68a0003e90 tx=0x7f68a0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd19c107773c058a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.955+0000 7f6893fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:10:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.956+0000 7f6893fff640 1 -- 172.21.15.110:0/3811506382 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a004e060 con 0x55eb7a025e50 2024-04-25T00:10:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.956+0000 7f6893fff640 1 -- 172.21.15.110:0/3811506382 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f68a0031800 con 0x55eb7a025e50 2024-04-25T00:10:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.956+0000 7f68bd770740 1 -- 172.21.15.110:0/3811506382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a0003aa0 con 0x55eb7a025e50 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.956+0000 7f6893fff640 1 -- 172.21.15.110:0/3811506382 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68a0035070 con 0x55eb7a025e50 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.956+0000 7f68bd770740 10 client.15942 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.956+0000 7f68bd770740 1 -- 172.21.15.110:0/3811506382 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eb7a02e900 con 0x55eb7a025e50 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.957+0000 7f6893fff640 1 -- 172.21.15.110:0/3811506382 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a0003aa0 con 0x55eb7a025e50 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f6893fff640 1 -- 172.21.15.110:0/3811506382 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f68a003d850 con 0x55eb7a025e50 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f68bd770740 10 client.15942 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f68bd770740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f68bd770740 20 client.15942 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f68bd770740 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eb7a03c1b0 0x55eb7a03e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f68bd770740 4 client.15942 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f68bd770740 1 -- 172.21.15.110:0/3811506382 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb79f971f0 con 0x55eb7a03c1b0 2024-04-25T00:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f68b8802640 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eb7a03c1b0 0x55eb7a03e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f6893fff640 10 client.15942 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.958+0000 7f68b8802640 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eb7a03c1b0 0x55eb7a03e610 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0df214e4e8d8ecf server_cookie=51c9995d4dc28805 in_seq=0 out_seq=0 2024-04-25T00:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.959+0000 7f6893fff640 1 -- 172.21.15.110:0/3811506382 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb79f971f0 con 0x55eb7a03c1b0 2024-04-25T00:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.959+0000 7f6893fff640 10 client.15942 handle_command_reply: tid=0 2024-04-25T00:10:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 client.15942 shutdown 2024-04-25T00:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 -- 172.21.15.110:0/3811506382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 msgr2=0x55eb7a0302d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a0302d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f68a0003e90 tx=0x7f68a0005850 comp rx=0 tx=0).stop 2024-04-25T00:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 -- 172.21.15.110:0/3811506382 shutdown_connections 2024-04-25T00:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eb7a03c1b0 0x55eb7a03e610 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a02b9b0 0x55eb7a037850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a030810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 --2- 172.21.15.110:0/3811506382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a0302d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.960+0000 7f68bd770740 1 -- 172.21.15.110:0/3811506382 >> 172.21.15.110:0/3811506382 conn(0x55eb7a0214c0 msgr2=0x55eb7a0205c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.961+0000 7f6893fff640 0 client.15942 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.961+0000 7f68bd770740 1 -- 172.21.15.110:0/3811506382 shutdown_connections 2024-04-25T00:10:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.961+0000 7f68bd770740 1 -- 172.21.15.110:0/3811506382 wait complete. 2024-04-25T00:10:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.962+0000 7f68bd770740 20 client.15942 trim_cache size 0 max 16384 2024-04-25T00:10:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.985+0000 7f68bd770740 1 Processor -- start 2024-04-25T00:10:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.985+0000 7f68bd770740 1 -- start start 2024-04-25T00:10:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.985+0000 7f68bd770740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a024470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68bd770740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a026840 0x55eb7a0377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68bd770740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68bd770740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b1920 con 0x55eb7a025e50 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68bd770740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41af1f0 con 0x55eb7a026840 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68b3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68b3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0249b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37842/0 (socket says 172.21.15.110:37842) 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68b3fff640 1 -- 172.21.15.110:0/1042507885 learned_addr learned my addr 172.21.15.110:0/1042507885 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68bd770740 1 -- 172.21.15.110:0/1042507885 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ad040 con 0x55eb7a02b9b0 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.986+0000 7f68b3fff640 1 --2- 172.21.15.110:0/1042507885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a024470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.987+0000 7f68b3fff640 1 -- 172.21.15.110:0/1042507885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a026840 msgr2=0x55eb7a0377e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.987+0000 7f68b3fff640 1 --2- 172.21.15.110:0/1042507885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a026840 0x55eb7a0377e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.987+0000 7f68b3fff640 1 -- 172.21.15.110:0/1042507885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 msgr2=0x55eb7a024470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.987+0000 7f68b3fff640 1 --2- 172.21.15.110:0/1042507885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a024470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.987+0000 7f68b3fff640 1 -- 172.21.15.110:0/1042507885 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7a02ebe0 con 0x55eb7a02b9b0 2024-04-25T00:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.987+0000 7f68b3fff640 1 --2- 172.21.15.110:0/1042507885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0249b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68a0005940 tx=0x7f68a0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=655f4a171076dfe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.988+0000 7f689e7fc640 1 -- 172.21.15.110:0/1042507885 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0059040 con 0x55eb7a02b9b0 2024-04-25T00:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.988+0000 7f689e7fc640 1 -- 172.21.15.110:0/1042507885 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a003d900 con 0x55eb7a02b9b0 2024-04-25T00:10:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.988+0000 7f689e7fc640 1 -- 172.21.15.110:0/1042507885 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a004e040 con 0x55eb7a02b9b0 2024-04-25T00:10:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.992+0000 7f68bd770740 1 -- 172.21.15.110:0/1042507885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a02b9b0 msgr2=0x55eb7a0249b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.992+0000 7f68bd770740 1 --2- 172.21.15.110:0/1042507885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0249b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68a0005940 tx=0x7f68a0005850 comp rx=0 tx=0).stop 2024-04-25T00:10:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 -- 172.21.15.110:0/1042507885 shutdown_connections 2024-04-25T00:10:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 --2- 172.21.15.110:0/1042507885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0249b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 --2- 172.21.15.110:0/1042507885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a026840 0x55eb7a0377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 --2- 172.21.15.110:0/1042507885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a025e50 0x55eb7a024470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 -- 172.21.15.110:0/1042507885 >> 172.21.15.110:0/1042507885 conn(0x55eb7a0214c0 msgr2=0x55eb7a02fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 -- 172.21.15.110:0/1042507885 shutdown_connections 2024-04-25T00:10:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 -- 172.21.15.110:0/1042507885 wait complete. 2024-04-25T00:10:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 -- start start 2024-04-25T00:10:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.993+0000 7f68bd770740 1 Processor -- start 2024-04-25T00:10:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68bd770740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a037320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68bd770740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a033e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68bd770740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68bd770740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ad6b0 con 0x55eb7a02b9b0 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68bd770740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41aeb20 con 0x55eb7a025e50 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68bd770740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41add80 con 0x55eb7a026840 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68bd770740 10 client.0 _ll_register_callbacks cb 0x7f68a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68b8802640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a033e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68b8802640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a033e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:37856/0 (socket says 172.21.15.110:37856) 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.994+0000 7f68b8802640 1 -- 172.21.15.110:0/1723690030 learned_addr learned my addr 172.21.15.110:0/1723690030 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f68b3fff640 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a037320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f68b3fff640 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f68b8802640 1 -- 172.21.15.110:0/1723690030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 msgr2=0x55eb7a037320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f68b8802640 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a037320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f68b8802640 1 -- 172.21.15.110:0/1723690030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 msgr2=0x55eb7a0343a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f68b8802640 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0343a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f68b8802640 1 -- 172.21.15.110:0/1723690030 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7a037950 con 0x55eb7a026840 2024-04-25T00:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f68b8802640 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a033e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68a8000c00 tx=0x7f68a8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=936dddbad63b2bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.995+0000 7f6893fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.996+0000 7f6893fff640 1 -- 172.21.15.110:0/1723690030 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8024070 con 0x55eb7a026840 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.996+0000 7f6893fff640 1 -- 172.21.15.110:0/1723690030 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f68a8020480 con 0x55eb7a026840 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.996+0000 7f68bd770740 1 -- 172.21.15.110:0/1723690030 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a003d900 con 0x55eb7a026840 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.996+0000 7f6893fff640 1 -- 172.21.15.110:0/1723690030 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68a802d050 con 0x55eb7a026840 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.996+0000 7f68bd770740 10 client.25504 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.996+0000 7f68bd770740 1 -- 172.21.15.110:0/1723690030 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eb7a0a1770 con 0x55eb7a026840 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.997+0000 7f6893fff640 1 -- 172.21.15.110:0/1723690030 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a003d900 con 0x55eb7a026840 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.997+0000 7f6893fff640 1 -- 172.21.15.110:0/1723690030 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f68a801fb10 con 0x55eb7a026840 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.997+0000 7f68bd770740 10 client.25504 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.997+0000 7f68bd770740 10 client.25504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.997+0000 7f68bd770740 20 client.25504 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.997+0000 7f68bd770740 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eb7a0a2150 0x55eb7a0a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.997+0000 7f68bd770740 4 client.25504 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:10:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.997+0000 7f68bd770740 1 -- 172.21.15.110:0/1723690030 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb79f91540 con 0x55eb7a0a2150 2024-04-25T00:10:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.998+0000 7f68b3fff640 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eb7a0a2150 0x55eb7a0a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.998+0000 7f68b3fff640 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eb7a0a2150 0x55eb7a0a4570 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329c31498e5f92f5 server_cookie=b8003ba0ce976978 in_seq=0 out_seq=0 2024-04-25T00:10:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:41.998+0000 7f6893fff640 10 client.25504 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:41 smithi188 ceph-mon[33285]: pgmap v610: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f6893fff640 1 -- 172.21.15.110:0/1723690030 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb79f91540 con 0x55eb7a0a2150 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f6893fff640 10 client.25504 handle_command_reply: tid=0 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 client.25504 shutdown 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 -- 172.21.15.110:0/1723690030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 msgr2=0x55eb7a033e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a033e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68a8000c00 tx=0x7f68a8010800 comp rx=0 tx=0).stop 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 -- 172.21.15.110:0/1723690030 shutdown_connections 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eb7a0a2150 0x55eb7a0a4570 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb7a02b9b0 0x55eb7a0343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f6893fff640 0 client.25504 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb7a026840 0x55eb7a033e60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 --2- 172.21.15.110:0/1723690030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb7a025e50 0x55eb7a037320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 -- 172.21.15.110:0/1723690030 >> 172.21.15.110:0/1723690030 conn(0x55eb7a0214c0 msgr2=0x55eb7a024470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 -- 172.21.15.110:0/1723690030 shutdown_connections 2024-04-25T00:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.020+0000 7f68bd770740 1 -- 172.21.15.110:0/1723690030 wait complete. 2024-04-25T00:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68bd770740 20 client.25504 trim_cache size 0 max 16384 2024-04-25T00:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 -- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f68a40a2cb0 msgr2=0x7f68a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f68a40a2cb0 0x7f68a40a5170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68ac098990 tx=0x7f68ac0bc040 comp rx=0 tx=0).stop 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 -- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4109980 msgr2=0x7f68b41a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4109980 0x7f68b41a5500 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00eb10 tx=0x7f68ac00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 -- 172.21.15.110:0/1467653678 shutdown_connections 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f68a40a2cb0 0x7f68a40a5170 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68b4109980 0x7f68b41a5500 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68b4108fc0 0x7f68b419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 --2- 172.21.15.110:0/1467653678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b4107dd0 0x7f68b419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.021+0000 7f68baa66640 1 -- 172.21.15.110:0/1467653678 >> 172.21.15.110:0/1467653678 conn(0x7f68b4076d90 msgr2=0x7f68b410c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.022+0000 7f68baa66640 1 -- 172.21.15.110:0/1467653678 shutdown_connections 2024-04-25T00:10:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:42.022+0000 7f68baa66640 1 -- 172.21.15.110:0/1467653678 wait complete. 2024-04-25T00:10:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:41 smithi110 ceph-mon[29187]: pgmap v610: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2024-04-25T00:10:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:41 smithi173 ceph-mon[32402]: pgmap v610: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 1.04k op/s 2024-04-25T00:10:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:43 smithi188 ceph-mon[33285]: pgmap v611: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.4 MiB/s wr, 977 op/s 2024-04-25T00:10:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:43 smithi110 ceph-mon[29187]: pgmap v611: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.4 MiB/s wr, 977 op/s 2024-04-25T00:10:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:43 smithi173 ceph-mon[32402]: pgmap v611: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.4 MiB/s wr, 977 op/s 2024-04-25T00:10:46.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:45 smithi188 ceph-mon[33285]: pgmap v612: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 1k op/s 2024-04-25T00:10:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:45 smithi110 ceph-mon[29187]: pgmap v612: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 1k op/s 2024-04-25T00:10:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:45 smithi173 ceph-mon[32402]: pgmap v612: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 1k op/s 2024-04-25T00:10:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 460.0 s, 834.0 tps, lat 119.853 ms stddev 18.686 2024-04-25T00:10:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:47 smithi188 ceph-mon[33285]: pgmap v613: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 743 op/s 2024-04-25T00:10:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:47 smithi110 ceph-mon[29187]: pgmap v613: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 743 op/s 2024-04-25T00:10:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:47 smithi173 ceph-mon[32402]: pgmap v613: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 743 op/s 2024-04-25T00:10:50.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:49 smithi188 ceph-mon[33285]: pgmap v614: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 1.03k op/s 2024-04-25T00:10:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:49 smithi110 ceph-mon[29187]: pgmap v614: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 1.03k op/s 2024-04-25T00:10:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:49 smithi173 ceph-mon[32402]: pgmap v614: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 1.03k op/s 2024-04-25T00:10:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 465.0 s, 890.8 tps, lat 112.333 ms stddev 11.647 2024-04-25T00:10:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:51 smithi188 ceph-mon[33285]: pgmap v615: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 861 op/s 2024-04-25T00:10:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:51 smithi110 ceph-mon[29187]: pgmap v615: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 861 op/s 2024-04-25T00:10:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:51 smithi173 ceph-mon[32402]: pgmap v615: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 861 op/s 2024-04-25T00:10:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:53 smithi188 ceph-mon[33285]: pgmap v616: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.1 MiB/s wr, 1.04k op/s 2024-04-25T00:10:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:53 smithi110 ceph-mon[29187]: pgmap v616: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.1 MiB/s wr, 1.04k op/s 2024-04-25T00:10:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:53 smithi173 ceph-mon[32402]: pgmap v616: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.1 MiB/s wr, 1.04k op/s 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/2338277135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 msgr2=0x7f5a88104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/2338277135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 0x7f5a88104430 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009860 tx=0x7f5a8402a340 comp rx=0 tx=0).stop 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/2338277135 shutdown_connections 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/2338277135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a881072a0 0x7f5a881096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/2338277135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88104970 0x7f5a88106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/2338277135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 0x7f5a88104430 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/2338277135 >> 172.21.15.110:0/2338277135 conn(0x7f5a880fba30 msgr2=0x7f5a880fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/2338277135 shutdown_connections 2024-04-25T00:10:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.339+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/2338277135 wait complete. 2024-04-25T00:10:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.340+0000 7f5a8eb9e640 1 Processor -- start 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.340+0000 7f5a8eb9e640 1 -- start start 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.340+0000 7f5a8eb9e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 0x7f5a881019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.340+0000 7f5a8eb9e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88104970 0x7f5a880ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.340+0000 7f5a8eb9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a881072a0 0x7f5a88100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.340+0000 7f5a8eb9e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810d100 con 0x7f5a88102040 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.340+0000 7f5a8eb9e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810cf80 con 0x7f5a88104970 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.340+0000 7f5a8eb9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810d280 con 0x7f5a881072a0 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 0x7f5a881019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8d39b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88104970 0x7f5a880ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 0x7f5a881019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49894/0 (socket says 172.21.15.110:49894) 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 -- 172.21.15.110:0/1896805848 learned_addr learned my addr 172.21.15.110:0/1896805848 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8e39d640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a881072a0 0x7f5a88100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 -- 172.21.15.110:0/1896805848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a881072a0 msgr2=0x7f5a88100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a881072a0 0x7f5a88100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 -- 172.21.15.110:0/1896805848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88104970 msgr2=0x7f5a880ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88104970 0x7f5a880ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 -- 172.21.15.110:0/1896805848 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a88100dc0 con 0x7f5a88102040 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88104970 0x7f5a880ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 0x7f5a881019b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a84005940 tx=0x7f5a84030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a7affd640 1 -- 172.21.15.110:0/1896805848 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84037050 con 0x7f5a88102040 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/1896805848 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a881a1900 con 0x7f5a88102040 2024-04-25T00:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/1896805848 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a881a1ea0 con 0x7f5a88102040 2024-04-25T00:10:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a7affd640 1 -- 172.21.15.110:0/1896805848 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a84043070 con 0x7f5a88102040 2024-04-25T00:10:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.341+0000 7f5a7affd640 1 -- 172.21.15.110:0/1896805848 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84003c60 con 0x7f5a88102040 2024-04-25T00:10:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.344+0000 7f5a7affd640 1 -- 172.21.15.110:0/1896805848 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5a84004040 con 0x7f5a88102040 2024-04-25T00:10:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.345+0000 7f5a7affd640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5a740a2c00 0x7f5a740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.345+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5a740a2c00 0x7f5a740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.345+0000 7f5a7affd640 1 -- 172.21.15.110:0/1896805848 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5a840ef520 con 0x7f5a88102040 2024-04-25T00:10:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.345+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5a740a2c00 0x7f5a740a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5a88101720 tx=0x7f5a7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a92bed740 1 Processor -- start 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a92bed740 1 -- start start 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965030 0x55a006965450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965a20 0x55a00696a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00696d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a92bed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a0068c8cb0 con 0x55a00696ab90 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a92bed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a0068715b0 con 0x55a006965030 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a92bed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a0068c4c70 con 0x55a006965a20 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a8d39b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965a20 0x55a00696a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a8e39d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00696d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965030 0x55a006965450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965030 0x55a006965450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:60116/0 (socket says 172.21.15.110:60116) 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.364+0000 7f5a8db9c640 1 -- 172.21.15.110:0/4154286437 learned_addr learned my addr 172.21.15.110:0/4154286437 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a8e39d640 1 -- 172.21.15.110:0/4154286437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965a20 msgr2=0x55a00696a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a8e39d640 1 --2- 172.21.15.110:0/4154286437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965a20 0x55a00696a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a8e39d640 1 -- 172.21.15.110:0/4154286437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965030 msgr2=0x55a006965450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a8e39d640 1 --2- 172.21.15.110:0/4154286437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965030 0x55a006965450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a8e39d640 1 -- 172.21.15.110:0/4154286437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0065dd210 con 0x55a00696ab90 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/4154286437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965030 0x55a006965450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a8e39d640 1 --2- 172.21.15.110:0/4154286437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00696d020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009f00 tx=0x7f5a80011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530f5c99ded4ee50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a73fff640 1 -- 172.21.15.110:0/4154286437 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80004d20 con 0x55a00696ab90 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a73fff640 1 -- 172.21.15.110:0/4154286437 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a80004030 con 0x55a00696ab90 2024-04-25T00:10:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.365+0000 7f5a73fff640 1 -- 172.21.15.110:0/4154286437 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80005280 con 0x55a00696ab90 2024-04-25T00:10:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.366+0000 7f5a92bed740 1 -- 172.21.15.110:0/4154286437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 msgr2=0x55a00696d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.366+0000 7f5a92bed740 1 --2- 172.21.15.110:0/4154286437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00696d020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009f00 tx=0x7f5a80011810 comp rx=0 tx=0).stop 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.366+0000 7f5a92bed740 1 -- 172.21.15.110:0/4154286437 shutdown_connections 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.366+0000 7f5a92bed740 1 --2- 172.21.15.110:0/4154286437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00696d020 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.366+0000 7f5a92bed740 1 --2- 172.21.15.110:0/4154286437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965a20 0x55a00696a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.366+0000 7f5a92bed740 1 --2- 172.21.15.110:0/4154286437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965030 0x55a006965450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.366+0000 7f5a92bed740 1 -- 172.21.15.110:0/4154286437 >> 172.21.15.110:0/4154286437 conn(0x55a0069606a0 msgr2=0x55a006960aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.367+0000 7f5a92bed740 1 -- 172.21.15.110:0/4154286437 shutdown_connections 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.367+0000 7f5a92bed740 1 -- 172.21.15.110:0/4154286437 wait complete. 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.367+0000 7f5a92bed740 1 -- start start 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.368+0000 7f5a92bed740 1 Processor -- start 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.368+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006977c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.368+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a0069781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.368+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 0x55a00697f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006977c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006977c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:59210/0 (socket says 172.21.15.110:59210) 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a8db9c640 1 -- 172.21.15.110:0/2975900759 learned_addr learned my addr 172.21.15.110:0/2975900759 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a8e39d640 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 0x55a00697f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a00696db40 con 0x55a006965a20 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a0069781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a006884b50 con 0x55a00696ab90 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a00696dcc0 con 0x55a006965030 2024-04-25T00:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.369+0000 7f5a92bed740 10 client.0 _ll_register_callbacks cb 0x55a0068bfca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a8db9c640 1 -- 172.21.15.110:0/2975900759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 msgr2=0x55a00697f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 0x55a00697f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a8db9c640 1 -- 172.21.15.110:0/2975900759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 msgr2=0x55a0069781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a0069781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a8db9c640 1 -- 172.21.15.110:0/2975900759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0066a9620 con 0x55a006965030 2024-04-25T00:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a0069781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006977c60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009980 tx=0x7f5a840bb970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3229c90761f3a96c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a717fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a717fa640 1 -- 172.21.15.110:0/2975900759 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a840bc4b0 con 0x55a006965030 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a80004030 con 0x55a006965030 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a717fa640 1 -- 172.21.15.110:0/2975900759 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5a840b8420 con 0x55a006965030 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a717fa640 1 -- 172.21.15.110:0/2975900759 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a840bbdb0 con 0x55a006965030 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a92bed740 10 client.25516 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.370+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a00696db40 con 0x55a006965030 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a717fa640 1 -- 172.21.15.110:0/2975900759 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a80004030 con 0x55a006965030 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a717fa640 1 -- 172.21.15.110:0/2975900759 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5a840ef0b0 con 0x55a006965030 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a92bed740 10 client.25516 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a92bed740 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a92bed740 20 client.25516 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a006983e40 0x55a0069862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a92bed740 4 client.25516 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0068d61f0 con 0x55a006983e40 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a006983e40 0x55a0069862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a006983e40 0x55a0069862e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c234f50d3e6bc7 server_cookie=2e40a065f0c3423a in_seq=0 out_seq=0 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.371+0000 7f5a717fa640 10 client.25516 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.372+0000 7f5a717fa640 1 -- 172.21.15.110:0/2975900759 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0068d61f0 con 0x55a006983e40 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.372+0000 7f5a717fa640 10 client.25516 handle_command_reply: tid=0 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 client.25516 shutdown 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 msgr2=0x55a006977c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006977c60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009980 tx=0x7f5a840bb970 comp rx=0 tx=0).stop 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 shutdown_connections 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a006983e40 0x55a0069862e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 0x55a00697f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a717fa640 0 client.25516 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a0069781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2975900759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006977c60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 >> 172.21.15.110:0/2975900759 conn(0x55a0069606a0 msgr2=0x55a00695f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 shutdown_connections 2024-04-25T00:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 1 -- 172.21.15.110:0/2975900759 wait complete. 2024-04-25T00:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.373+0000 7f5a92bed740 20 client.25516 trim_cache size 0 max 16384 2024-04-25T00:10:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.402+0000 7f5a92bed740 1 Processor -- start 2024-04-25T00:10:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.402+0000 7f5a92bed740 1 -- start start 2024-04-25T00:10:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.402+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006978100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.402+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a00697f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.402+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 0x55a00695b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.402+0000 7f5a92bed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c010c70 con 0x55a006965a20 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.402+0000 7f5a92bed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c00e540 con 0x55a00696ab90 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.402+0000 7f5a92bed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c00c390 con 0x55a006965030 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006978100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006978100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:59218/0 (socket says 172.21.15.110:59218) 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 -- 172.21.15.110:0/1578301221 learned_addr learned my addr 172.21.15.110:0/1578301221 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a00697f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8e39d640 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 0x55a00695b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 -- 172.21.15.110:0/1578301221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 msgr2=0x55a00695b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 0x55a00695b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 -- 172.21.15.110:0/1578301221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 msgr2=0x55a00697f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a00697f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 -- 172.21.15.110:0/1578301221 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a006986960 con 0x55a006965030 2024-04-25T00:10:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.403+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006978100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009830 tx=0x7f5a840ba2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5051f4972817536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.404+0000 7f5a737fe640 1 -- 172.21.15.110:0/1578301221 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a840bc4b0 con 0x55a006965030 2024-04-25T00:10:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.404+0000 7f5a737fe640 1 -- 172.21.15.110:0/1578301221 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a840bf030 con 0x55a006965030 2024-04-25T00:10:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.404+0000 7f5a737fe640 1 -- 172.21.15.110:0/1578301221 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a840b7e30 con 0x55a006965030 2024-04-25T00:10:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.406+0000 7f5a92bed740 1 -- 172.21.15.110:0/1578301221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 msgr2=0x55a006978100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.406+0000 7f5a92bed740 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006978100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009830 tx=0x7f5a840ba2c0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.406+0000 7f5a92bed740 1 -- 172.21.15.110:0/1578301221 shutdown_connections 2024-04-25T00:10:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.406+0000 7f5a92bed740 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a00696ab90 0x55a00695b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.406+0000 7f5a92bed740 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a006965a20 0x55a00697f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.406+0000 7f5a92bed740 1 --2- 172.21.15.110:0/1578301221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a006978100 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.406+0000 7f5a92bed740 1 -- 172.21.15.110:0/1578301221 >> 172.21.15.110:0/1578301221 conn(0x55a0069606a0 msgr2=0x55a00695f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.407+0000 7f5a92bed740 1 -- 172.21.15.110:0/1578301221 shutdown_connections 2024-04-25T00:10:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.407+0000 7f5a92bed740 1 -- 172.21.15.110:0/1578301221 wait complete. 2024-04-25T00:10:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.407+0000 7f5a92bed740 1 -- start start 2024-04-25T00:10:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.407+0000 7f5a92bed740 1 Processor -- start 2024-04-25T00:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.408+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a00697c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.408+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965a20 0x55a00697c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.408+0000 7f5a92bed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00697d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.408+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a00697c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.408+0000 7f5a8db9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a00697c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:59224/0 (socket says 172.21.15.110:59224) 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.408+0000 7f5a8db9c640 1 -- 172.21.15.110:0/2749722914 learned_addr learned my addr 172.21.15.110:0/2749722914 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.408+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c00ca00 con 0x55a00696ab90 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.408+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c00de70 con 0x55a006965a20 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c00d0d0 con 0x55a006965030 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965a20 0x55a00697c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a92bed740 10 client.0 _ll_register_callbacks cb 0x7f5a840bbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a8e39d640 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00697d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a8db9c640 1 -- 172.21.15.110:0/2749722914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965a20 msgr2=0x55a00697c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965a20 0x55a00697c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a8db9c640 1 -- 172.21.15.110:0/2749722914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 msgr2=0x55a00697d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00697d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a8db9c640 1 -- 172.21.15.110:0/2749722914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00697df30 con 0x55a006965030 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a8db9c640 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a00697c3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5a840c44f0 tx=0x7f5a84005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e29f997b2dc5520 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.409+0000 7f5a717fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.410+0000 7f5a717fa640 1 -- 172.21.15.110:0/2749722914 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a840b7ba0 con 0x55a006965030 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.410+0000 7f5a717fa640 1 -- 172.21.15.110:0/2749722914 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5a840ef0d0 con 0x55a006965030 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.410+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a840bf030 con 0x55a006965030 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.410+0000 7f5a717fa640 1 -- 172.21.15.110:0/2749722914 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a840c4a20 con 0x55a006965030 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.410+0000 7f5a92bed740 10 client.25522 fetch_fsmap learned FSMap version 32 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.410+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a00697df30 con 0x55a006965030 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.410+0000 7f5a717fa640 1 -- 172.21.15.110:0/2749722914 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a840bf030 con 0x55a006965030 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.411+0000 7f5a717fa640 1 -- 172.21.15.110:0/2749722914 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5a840c12f0 con 0x55a006965030 2024-04-25T00:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.411+0000 7f5a92bed740 10 client.25522 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.411+0000 7f5a92bed740 10 client.25522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.411+0000 7f5a92bed740 20 client.25522 populate_metadata read hostname 'smithi110' 2024-04-25T00:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.411+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a0069ea460 0x55a0069ec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.411+0000 7f5a92bed740 4 client.25522 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.411+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0068d0540 con 0x55a0069ea460 2024-04-25T00:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.411+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a0069ea460 0x55a0069ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.412+0000 7f5a8d39b640 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a0069ea460 0x55a0069ec850 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423423f0d177bf86 server_cookie=d7d6618b17e5c110 in_seq=0 out_seq=0 2024-04-25T00:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.412+0000 7f5a717fa640 10 client.25522 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.431+0000 7f5a717fa640 1 -- 172.21.15.110:0/2749722914 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0068d0540 con 0x55a0069ea460 2024-04-25T00:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a717fa640 10 client.25522 handle_command_reply: tid=0 2024-04-25T00:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 client.25522 shutdown 2024-04-25T00:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 msgr2=0x55a00697c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a00697c3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5a840c44f0 tx=0x7f5a84005bd0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 shutdown_connections 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55a0069ea460 0x55a0069ec850 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a00696ab90 0x55a00697d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a006965a20 0x55a00697c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 --2- 172.21.15.110:0/2749722914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a006965030 0x55a00697c3f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 >> 172.21.15.110:0/2749722914 conn(0x55a0069606a0 msgr2=0x55a00695f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a717fa640 0 client.25522 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 shutdown_connections 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 1 -- 172.21.15.110:0/2749722914 wait complete. 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.432+0000 7f5a92bed740 20 client.25522 trim_cache size 0 max 16384 2024-04-25T00:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:10:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:10:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:10:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:10:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5a740a2c00 msgr2=0x7f5a740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5a740a2c00 0x7f5a740a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5a88101720 tx=0x7f5a7c009260 comp rx=0 tx=0).stop 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 msgr2=0x7f5a881019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 0x7f5a881019b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a84005940 tx=0x7f5a84030e00 comp rx=0 tx=0).stop 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/1896805848 shutdown_connections 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5a740a2c00 0x7f5a740a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a881072a0 0x7f5a88100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88104970 0x7f5a880ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 --2- 172.21.15.110:0/1896805848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a88102040 0x7f5a881019b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/1896805848 >> 172.21.15.110:0/1896805848 conn(0x7f5a880fba30 msgr2=0x7f5a88103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/1896805848 shutdown_connections 2024-04-25T00:10:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:10:55.433+0000 7f5a8eb9e640 1 -- 172.21.15.110:0/1896805848 wait complete. 2024-04-25T00:10:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:55 smithi110 ceph-mon[29187]: pgmap v617: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.6 MiB/s wr, 989 op/s 2024-04-25T00:10:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:55 smithi173 ceph-mon[32402]: pgmap v617: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.6 MiB/s wr, 989 op/s 2024-04-25T00:10:56.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:55 smithi188 ceph-mon[33285]: pgmap v617: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.6 MiB/s wr, 989 op/s 2024-04-25T00:10:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 470.0 s, 897.0 tps, lat 111.534 ms stddev 16.638 2024-04-25T00:10:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:57 smithi110 ceph-mon[29187]: pgmap v618: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.1 MiB/s wr, 814 op/s 2024-04-25T00:10:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:57 smithi173 ceph-mon[32402]: pgmap v618: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.1 MiB/s wr, 814 op/s 2024-04-25T00:10:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:57 smithi188 ceph-mon[33285]: pgmap v618: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.1 MiB/s wr, 814 op/s 2024-04-25T00:11:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:10:59 smithi110 ceph-mon[29187]: pgmap v619: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 1.20k op/s 2024-04-25T00:11:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:10:59 smithi173 ceph-mon[32402]: pgmap v619: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 1.20k op/s 2024-04-25T00:11:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:10:59 smithi188 ceph-mon[33285]: pgmap v619: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 1.20k op/s 2024-04-25T00:11:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 475.0 s, 856.0 tps, lat 116.878 ms stddev 17.574 2024-04-25T00:11:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:01 smithi110 ceph-mon[29187]: pgmap v620: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 MiB/s wr, 963 op/s 2024-04-25T00:11:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:01 smithi173 ceph-mon[32402]: pgmap v620: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 MiB/s wr, 963 op/s 2024-04-25T00:11:02.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:01 smithi188 ceph-mon[33285]: pgmap v620: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 MiB/s wr, 963 op/s 2024-04-25T00:11:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:03 smithi110 ceph-mon[29187]: pgmap v621: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-04-25T00:11:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:03 smithi173 ceph-mon[32402]: pgmap v621: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-04-25T00:11:04.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:03 smithi188 ceph-mon[33285]: pgmap v621: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-04-25T00:11:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:05 smithi110 ceph-mon[29187]: pgmap v622: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 975 op/s 2024-04-25T00:11:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:05 smithi173 ceph-mon[32402]: pgmap v622: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 975 op/s 2024-04-25T00:11:06.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:05 smithi188 ceph-mon[33285]: pgmap v622: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 975 op/s 2024-04-25T00:11:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 480.0 s, 818.0 tps, lat 122.078 ms stddev 16.368 2024-04-25T00:11:07.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:06 smithi110 ceph-mon[29187]: pgmap v623: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 877 op/s 2024-04-25T00:11:07.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:06 smithi173 ceph-mon[32402]: pgmap v623: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 877 op/s 2024-04-25T00:11:07.168 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:06 smithi188 ceph-mon[33285]: pgmap v623: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 877 op/s 2024-04-25T00:11:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.794+0000 7f7fb77fe640 1 -- 172.21.15.110:0/2118094571 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0035020 con 0x7f7fb8102250 2024-04-25T00:11:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.796+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2118094571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 msgr2=0x7f7fb8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.796+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2118094571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 0x7f7fb8108ae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009900 tx=0x7f7fa002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.798+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2118094571 shutdown_connections 2024-04-25T00:11:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.798+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2118094571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 0x7f7fb8108ae0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.798+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2118094571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb8101890 0x7f7fb8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.798+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2118094571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fb81006a0 0x7f7fb8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.798+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2118094571 >> 172.21.15.110:0/2118094571 conn(0x7f7fb80fbc10 msgr2=0x7f7fb80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.798+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2118094571 shutdown_connections 2024-04-25T00:11:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.798+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2118094571 wait complete. 2024-04-25T00:11:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.799+0000 7f7fbf15a640 1 Processor -- start 2024-04-25T00:11:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.799+0000 7f7fbf15a640 1 -- start start 2024-04-25T00:11:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.799+0000 7f7fbf15a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb81006a0 0x7f7fb819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.799+0000 7f7fbf15a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fb8101890 0x7f7fb819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.799+0000 7f7fbf15a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 0x7f7fb81a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.799+0000 7f7fbf15a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810d2a0 con 0x7f7fb81006a0 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.799+0000 7f7fbf15a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810d120 con 0x7f7fb8101890 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.799+0000 7f7fbf15a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810d420 con 0x7f7fb8102250 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbcef6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb81006a0 0x7f7fb819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbd6f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 0x7f7fb81a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbcef6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb81006a0 0x7f7fb819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57002/0 (socket says 172.21.15.110:57002) 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbcef6640 1 -- 172.21.15.110:0/2529467635 learned_addr learned my addr 172.21.15.110:0/2529467635 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fb8101890 0x7f7fb819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbd6f7640 1 -- 172.21.15.110:0/2529467635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fb8101890 msgr2=0x7f7fb819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbd6f7640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fb8101890 0x7f7fb819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbd6f7640 1 -- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb81006a0 msgr2=0x7f7fb819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbd6f7640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb81006a0 0x7f7fb819df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbd6f7640 1 -- 172.21.15.110:0/2529467635 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb81a5b10 con 0x7f7fb8102250 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.800+0000 7f7fbd6f7640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 0x7f7fb81a54a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002a910 tx=0x7f7fa0032d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.801+0000 7f7fb5ffb640 1 -- 172.21.15.110:0/2529467635 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa003f030 con 0x7f7fb8102250 2024-04-25T00:11:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.801+0000 7f7fb5ffb640 1 -- 172.21.15.110:0/2529467635 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fa0003cb0 con 0x7f7fb8102250 2024-04-25T00:11:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.801+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2529467635 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb81a5da0 con 0x7f7fb8102250 2024-04-25T00:11:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.801+0000 7f7fb5ffb640 1 -- 172.21.15.110:0/2529467635 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0035070 con 0x7f7fb8102250 2024-04-25T00:11:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.801+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2529467635 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb81a6280 con 0x7f7fb8102250 2024-04-25T00:11:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.804+0000 7f7fc1e63740 1 Processor -- start 2024-04-25T00:11:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.804+0000 7f7fb5ffb640 1 -- 172.21.15.110:0/2529467635 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7fa0042030 con 0x7f7fb8102250 2024-04-25T00:11:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fc1e63740 1 -- start start 2024-04-25T00:11:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8effee50 0x559a8efff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f004340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f006e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fc1e63740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559a8ef62cb0 con 0x559a8f0049b0 2024-04-25T00:11:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fc1e63740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559a8ef0b5b0 con 0x559a8efff840 2024-04-25T00:11:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fc1e63740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559a8ef5ec70 con 0x559a8effee50 2024-04-25T00:11:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8effee50 0x559a8efff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fbcef6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f004340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8effee50 0x559a8efff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:41792/0 (socket says 172.21.15.110:41792) 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 -- 172.21.15.110:0/1847381797 learned_addr learned my addr 172.21.15.110:0/1847381797 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f006e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 -- 172.21.15.110:0/1847381797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 msgr2=0x559a8f004340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f004340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 -- 172.21.15.110:0/1847381797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 msgr2=0x559a8f006e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f006e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fb7fff640 1 -- 172.21.15.110:0/1847381797 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8ec77210 con 0x559a8effee50 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.805+0000 7f7fbcef6640 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f004340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.806+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8effee50 0x559a8efff270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005f40 tx=0x7f7fb0005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b0aaf26a8cd2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.806+0000 7f7faeffd640 1 -- 172.21.15.110:0/1847381797 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000fc50 con 0x559a8effee50 2024-04-25T00:11:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.806+0000 7f7faeffd640 1 -- 172.21.15.110:0/1847381797 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fb0003e20 con 0x559a8effee50 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.806+0000 7f7fb5ffb640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7fa40a2cb0 0x7f7fa40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fbcef6640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7fa40a2cb0 0x7f7fa40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fb5ffb640 1 -- 172.21.15.110:0/2529467635 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7fa00ef380 con 0x7f7fb8102250 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1847381797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8effee50 msgr2=0x559a8efff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8effee50 0x559a8efff270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005f40 tx=0x7f7fb0005bd0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7faeffd640 1 -- 172.21.15.110:0/1847381797 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0004820 con 0x559a8effee50 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1847381797 shutdown_connections 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f006e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f004340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/1847381797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8effee50 0x559a8efff270 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1847381797 >> 172.21.15.110:0/1847381797 conn(0x559a8effa4c0 msgr2=0x559a8effa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.807+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1847381797 shutdown_connections 2024-04-25T00:11:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.808+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1847381797 wait complete. 2024-04-25T00:11:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.808+0000 7f7fbcef6640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7fa40a2cb0 0x7f7fa40a5170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x559a8efffc40 tx=0x7f7fa8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.808+0000 7f7fc1e63740 1 -- start start 2024-04-25T00:11:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.808+0000 7f7fc1e63740 1 Processor -- start 2024-04-25T00:11:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f009370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f0108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fc1e63740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559a8f0079b0 con 0x559a8f0049b0 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fc1e63740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559a8ef1eb50 con 0x559a8effee50 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fb7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f0098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fbcef6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f009370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fb7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f0098b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:41802/0 (socket says 172.21.15.110:41802) 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fb7fff640 1 -- 172.21.15.110:0/2970767135 learned_addr learned my addr 172.21.15.110:0/2970767135 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559a8f007b30 con 0x559a8efff840 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fbd6f7640 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f0108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.809+0000 7f7fc1e63740 10 client.0 _ll_register_callbacks cb 0x559a8ef59ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7fb7fff640 1 -- 172.21.15.110:0/2970767135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 msgr2=0x559a8f009370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f009370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7fb7fff640 1 -- 172.21.15.110:0/2970767135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 msgr2=0x559a8f0108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f0108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7fb7fff640 1 -- 172.21.15.110:0/2970767135 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8ed43620 con 0x559a8efff840 2024-04-25T00:11:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f0098b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005c40 tx=0x7f7fb000fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f4defa52e62eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7f97fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7f97fff640 1 -- 172.21.15.110:0/2970767135 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000fe40 con 0x559a8efff840 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb0003e20 con 0x559a8efff840 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.810+0000 7f7f97fff640 1 -- 172.21.15.110:0/2970767135 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7fb00047d0 con 0x559a8efff840 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7f97fff640 1 -- 172.21.15.110:0/2970767135 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7fb0003970 con 0x559a8efff840 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fc1e63740 10 client.25534 fetch_fsmap learned FSMap version 32 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a8f013c40 con 0x559a8efff840 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7f97fff640 1 -- 172.21.15.110:0/2970767135 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fb0003e20 con 0x559a8efff840 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7f97fff640 1 -- 172.21.15.110:0/2970767135 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7fb001d030 con 0x559a8efff840 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fc1e63740 10 client.25534 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fc1e63740 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fc1e63740 20 client.25534 populate_metadata read hostname 'smithi110' 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559a8f015050 0x559a8f017480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fc1e63740 4 client.25534 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a8ef701f0 con 0x559a8f015050 2024-04-25T00:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.811+0000 7f7fbcef6640 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559a8f015050 0x559a8f017480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.812+0000 7f7fbcef6640 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559a8f015050 0x559a8f017480 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30bac7f32a4fda86 server_cookie=df4bf42be9ea8e41 in_seq=0 out_seq=0 2024-04-25T00:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.812+0000 7f7f97fff640 10 client.25534 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7f97fff640 1 -- 172.21.15.110:0/2970767135 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a8ef701f0 con 0x559a8f015050 2024-04-25T00:11:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7f97fff640 10 client.25534 handle_command_reply: tid=0 2024-04-25T00:11:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 client.25534 shutdown 2024-04-25T00:11:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 msgr2=0x559a8f0098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f0098b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005c40 tx=0x7f7fb000fc50 comp rx=0 tx=0).stop 2024-04-25T00:11:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 shutdown_connections 2024-04-25T00:11:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559a8f015050 0x559a8f017480 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7f97fff640 0 client.25534 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f0108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f0098b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2970767135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f009370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 >> 172.21.15.110:0/2970767135 conn(0x559a8effa4c0 msgr2=0x559a8eff95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 shutdown_connections 2024-04-25T00:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2970767135 wait complete. 2024-04-25T00:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.813+0000 7f7fc1e63740 20 client.25534 trim_cache size 0 max 16384 2024-04-25T00:11:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.840+0000 7f7fc1e63740 1 Processor -- start 2024-04-25T00:11:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.840+0000 7f7fc1e63740 1 -- start start 2024-04-25T00:11:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.840+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8effee50 0x559a8effdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.841+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f0108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.841+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8f0049b0 0x559a8f01bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.841+0000 7f7fc1e63740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81b1930 con 0x559a8effee50 2024-04-25T00:11:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.841+0000 7f7fb7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f0108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.841+0000 7f7fb7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f0108b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:49546/0 (socket says 172.21.15.110:49546) 2024-04-25T00:11:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.841+0000 7f7fb7fff640 1 -- 172.21.15.110:0/1973222914 learned_addr learned my addr 172.21.15.110:0/1973222914 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.841+0000 7f7fc1e63740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81af200 con 0x559a8efff840 2024-04-25T00:11:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.841+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1973222914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81ad050 con 0x559a8f0049b0 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fb7fff640 1 -- 172.21.15.110:0/1973222914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8f0049b0 msgr2=0x559a8f01bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/1973222914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8f0049b0 0x559a8f01bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fb7fff640 1 -- 172.21.15.110:0/1973222914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8effee50 msgr2=0x559a8effdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/1973222914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8effee50 0x559a8effdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fb7fff640 1 -- 172.21.15.110:0/1973222914 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8f009c20 con 0x559a8efff840 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/1973222914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f0108b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000a010 tx=0x7f7fb0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e9b6b6e211d373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fae7fc640 1 -- 172.21.15.110:0/1973222914 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00043a0 con 0x559a8efff840 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fae7fc640 1 -- 172.21.15.110:0/1973222914 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fb0036020 con 0x559a8efff840 2024-04-25T00:11:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.842+0000 7f7fae7fc640 1 -- 172.21.15.110:0/1973222914 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0005920 con 0x559a8efff840 2024-04-25T00:11:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1973222914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 msgr2=0x559a8f0108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/1973222914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f0108b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000a010 tx=0x7f7fb0009580 comp rx=0 tx=0).stop 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1973222914 shutdown_connections 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/1973222914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8f0049b0 0x559a8f01bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/1973222914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8efff840 0x559a8f0108b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/1973222914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8effee50 0x559a8effdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1973222914 >> 172.21.15.110:0/1973222914 conn(0x559a8effa4c0 msgr2=0x559a8effd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1973222914 shutdown_connections 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 -- 172.21.15.110:0/1973222914 wait complete. 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 -- start start 2024-04-25T00:11:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.844+0000 7f7fc1e63740 1 Processor -- start 2024-04-25T00:11:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f00e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f00e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fc1e63740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f00f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fbcef6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f00e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fbcef6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f00e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:49554/0 (socket says 172.21.15.110:49554) 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fc1e63740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81ad6c0 con 0x559a8f0049b0 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fbcef6640 1 -- 172.21.15.110:0/2398992274 learned_addr learned my addr 172.21.15.110:0/2398992274 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81aeb30 con 0x559a8effee50 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81add90 con 0x559a8efff840 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fc1e63740 10 client.0 _ll_register_callbacks cb 0x7f7fb0011690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f00e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.845+0000 7f7fbd6f7640 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f00f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.846+0000 7f7fbcef6640 1 -- 172.21.15.110:0/2398992274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 msgr2=0x559a8f00e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.846+0000 7f7fbcef6640 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f00e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.846+0000 7f7fbcef6640 1 -- 172.21.15.110:0/2398992274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 msgr2=0x559a8f00f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.846+0000 7f7fbcef6640 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f00f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.846+0000 7f7fbcef6640 1 -- 172.21.15.110:0/2398992274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8f00fc60 con 0x559a8effee50 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.846+0000 7f7fbcef6640 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f00e150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x559a8f0002b0 tx=0x7f7fa8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b2a862dfd1c5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.846+0000 7f7f97fff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.847+0000 7f7f97fff640 1 -- 172.21.15.110:0/2398992274 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8031060 con 0x559a8effee50 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.847+0000 7f7f97fff640 1 -- 172.21.15.110:0/2398992274 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7fa800fbd0 con 0x559a8effee50 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.847+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb0036020 con 0x559a8effee50 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.847+0000 7f7f97fff640 1 -- 172.21.15.110:0/2398992274 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7fa800fdd0 con 0x559a8effee50 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.847+0000 7f7fc1e63740 10 client.25565 fetch_fsmap learned FSMap version 32 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.847+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a8f07b7b0 con 0x559a8effee50 2024-04-25T00:11:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.847+0000 7f7f97fff640 1 -- 172.21.15.110:0/2398992274 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fb0036020 con 0x559a8effee50 2024-04-25T00:11:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.848+0000 7f7f97fff640 1 -- 172.21.15.110:0/2398992274 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7fa8029a80 con 0x559a8effee50 2024-04-25T00:11:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.848+0000 7f7fc1e63740 10 client.25565 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:11:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.848+0000 7f7fc1e63740 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:11:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.848+0000 7f7fc1e63740 20 client.25565 populate_metadata read hostname 'smithi110' 2024-04-25T00:11:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.848+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559a8f015050 0x559a8f07e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.848+0000 7f7fc1e63740 4 client.25565 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:11:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.848+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a8ef6a540 con 0x559a8f015050 2024-04-25T00:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.848+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559a8f015050 0x559a8f07e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.849+0000 7f7fb7fff640 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559a8f015050 0x559a8f07e1b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f860af78c5c174e3 server_cookie=68791cceacc1b299 in_seq=0 out_seq=0 2024-04-25T00:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.849+0000 7f7f97fff640 10 client.25565 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.874+0000 7f7f97fff640 1 -- 172.21.15.110:0/2398992274 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a8ef6a540 con 0x559a8f015050 2024-04-25T00:11:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.874+0000 7f7f97fff640 10 client.25565 handle_command_reply: tid=0 2024-04-25T00:11:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.874+0000 7f7fc1e63740 1 client.25565 shutdown 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.874+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 msgr2=0x559a8f00e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.874+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f00e150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x559a8f0002b0 tx=0x7f7fa8004730 comp rx=0 tx=0).stop 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.874+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 shutdown_connections 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x559a8f015050 0x559a8f07e1b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a8f0049b0 0x559a8f00f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7f97fff640 0 client.25565 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a8efff840 0x559a8f00e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7fc1e63740 1 --2- 172.21.15.110:0/2398992274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559a8effee50 0x559a8f00e150 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 >> 172.21.15.110:0/2398992274 conn(0x559a8effa4c0 msgr2=0x559a8eff8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 shutdown_connections 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7fc1e63740 1 -- 172.21.15.110:0/2398992274 wait complete. 2024-04-25T00:11:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.875+0000 7f7fc1e63740 20 client.25565 trim_cache size 0 max 16384 2024-04-25T00:11:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:11:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:11:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:11:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:11:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7fa40a2cb0 msgr2=0x7f7fa40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7fa40a2cb0 0x7f7fa40a5170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x559a8efffc40 tx=0x7f7fa8005f90 comp rx=0 tx=0).stop 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2529467635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 msgr2=0x7f7fb81a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 0x7f7fb81a54a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002a910 tx=0x7f7fa0032d00 comp rx=0 tx=0).stop 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2529467635 shutdown_connections 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7fa40a2cb0 0x7f7fa40a5170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fb8102250 0x7f7fb81a54a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fb8101890 0x7f7fb819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 --2- 172.21.15.110:0/2529467635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fb81006a0 0x7f7fb819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2529467635 >> 172.21.15.110:0/2529467635 conn(0x7f7fb80fbc10 msgr2=0x7f7fb8109320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2529467635 shutdown_connections 2024-04-25T00:11:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:08.876+0000 7f7fbf15a640 1 -- 172.21.15.110:0/2529467635 wait complete. 2024-04-25T00:11:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:09 smithi188 ceph-mon[33285]: pgmap v624: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-04-25T00:11:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:09 smithi110 ceph-mon[29187]: pgmap v624: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-04-25T00:11:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:09 smithi173 ceph-mon[32402]: pgmap v624: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.6 MiB/s wr, 1.12k op/s 2024-04-25T00:11:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 485.0 s, 822.4 tps, lat 121.358 ms stddev 16.587 2024-04-25T00:11:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:11 smithi188 ceph-mon[33285]: pgmap v625: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 869 op/s 2024-04-25T00:11:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:11 smithi110 ceph-mon[29187]: pgmap v625: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 869 op/s 2024-04-25T00:11:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:11 smithi173 ceph-mon[32402]: pgmap v625: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 869 op/s 2024-04-25T00:11:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:13 smithi188 ceph-mon[33285]: pgmap v626: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.7 MiB/s wr, 1.01k op/s 2024-04-25T00:11:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:13 smithi110 ceph-mon[29187]: pgmap v626: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.7 MiB/s wr, 1.01k op/s 2024-04-25T00:11:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:13 smithi173 ceph-mon[32402]: pgmap v626: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.7 MiB/s wr, 1.01k op/s 2024-04-25T00:11:16.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:15 smithi188 ceph-mon[33285]: pgmap v627: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 1.01k op/s 2024-04-25T00:11:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:15 smithi110 ceph-mon[29187]: pgmap v627: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 1.01k op/s 2024-04-25T00:11:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:15 smithi173 ceph-mon[32402]: pgmap v627: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 1.01k op/s 2024-04-25T00:11:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 490.0 s, 871.0 tps, lat 115.117 ms stddev 14.966 2024-04-25T00:11:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:17 smithi188 ceph-mon[33285]: pgmap v628: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 838 op/s 2024-04-25T00:11:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:17 smithi110 ceph-mon[29187]: pgmap v628: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 838 op/s 2024-04-25T00:11:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:17 smithi173 ceph-mon[32402]: pgmap v628: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 838 op/s 2024-04-25T00:11:20.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:19 smithi188 ceph-mon[33285]: pgmap v629: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.0 MiB/s wr, 1.05k op/s 2024-04-25T00:11:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:19 smithi110 ceph-mon[29187]: pgmap v629: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.0 MiB/s wr, 1.05k op/s 2024-04-25T00:11:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:19 smithi173 ceph-mon[32402]: pgmap v629: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.0 MiB/s wr, 1.05k op/s 2024-04-25T00:11:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 495.0 s, 845.6 tps, lat 118.059 ms stddev 38.835 2024-04-25T00:11:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 -- 172.21.15.110:0/3706928694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed381006a0 msgr2=0x7fed38100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 --2- 172.21.15.110:0/3706928694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed381006a0 0x7fed38100ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fed28009860 tx=0x7fed2802a340 comp rx=0 tx=0).stop 2024-04-25T00:11:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 -- 172.21.15.110:0/3706928694 shutdown_connections 2024-04-25T00:11:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 --2- 172.21.15.110:0/3706928694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed38102250 0x7fed38108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 --2- 172.21.15.110:0/3706928694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed38101890 0x7fed38101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 --2- 172.21.15.110:0/3706928694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed381006a0 0x7fed38100ac0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 -- 172.21.15.110:0/3706928694 >> 172.21.15.110:0/3706928694 conn(0x7fed380fbc10 msgr2=0x7fed380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 -- 172.21.15.110:0/3706928694 shutdown_connections 2024-04-25T00:11:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.010+0000 7fed3f396640 1 -- 172.21.15.110:0/3706928694 wait complete. 2024-04-25T00:11:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.011+0000 7fed3f396640 1 Processor -- start 2024-04-25T00:11:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.012+0000 7fed3f396640 1 -- start start 2024-04-25T00:11:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.012+0000 7fed3f396640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed381006a0 0x7fed3810be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.012+0000 7fed3f396640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed38101890 0x7fed3810a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.012+0000 7fed3f396640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed38102250 0x7fed3810a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.012+0000 7fed3f396640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed3810d2a0 con 0x7fed381006a0 2024-04-25T00:11:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.012+0000 7fed3f396640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed3810d120 con 0x7fed38101890 2024-04-25T00:11:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.012+0000 7fed3f396640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed3810d420 con 0x7fed38102250 2024-04-25T00:11:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.013+0000 7fed3d132640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed381006a0 0x7fed3810be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.013+0000 7fed3d132640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed381006a0 0x7fed3810be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:32996/0 (socket says 172.21.15.110:32996) 2024-04-25T00:11:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.013+0000 7fed3d132640 1 -- 172.21.15.110:0/1396405124 learned_addr learned my addr 172.21.15.110:0/1396405124 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.013+0000 7fed3d933640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed38102250 0x7fed3810a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.013+0000 7fed3d132640 1 -- 172.21.15.110:0/1396405124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed38102250 msgr2=0x7fed3810a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.013+0000 7fed3d132640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed38102250 0x7fed3810a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.013+0000 7fed3c931640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed38101890 0x7fed3810a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.013+0000 7fed3d132640 1 -- 172.21.15.110:0/1396405124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed38101890 msgr2=0x7fed3810a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.014+0000 7fed3d132640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed38101890 0x7fed3810a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.014+0000 7fed3d132640 1 -- 172.21.15.110:0/1396405124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3819b870 con 0x7fed381006a0 2024-04-25T00:11:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.014+0000 7fed3c931640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed38101890 0x7fed3810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.014+0000 7fed3d132640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed381006a0 0x7fed3810be40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fed28005e00 tx=0x7fed28004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.015+0000 7fed2e7fc640 1 -- 172.21.15.110:0/1396405124 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28042020 con 0x7fed381006a0 2024-04-25T00:11:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.015+0000 7fed2e7fc640 1 -- 172.21.15.110:0/1396405124 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed28038040 con 0x7fed381006a0 2024-04-25T00:11:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.015+0000 7fed3f396640 1 -- 172.21.15.110:0/1396405124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3819bb00 con 0x7fed381006a0 2024-04-25T00:11:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.015+0000 7fed2e7fc640 1 -- 172.21.15.110:0/1396405124 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28037050 con 0x7fed381006a0 2024-04-25T00:11:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.015+0000 7fed3f396640 1 -- 172.21.15.110:0/1396405124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3819c040 con 0x7fed381006a0 2024-04-25T00:11:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:21 smithi188 ceph-mon[33285]: pgmap v630: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 MiB/s wr, 1.02k op/s 2024-04-25T00:11:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.018+0000 7fed42176740 1 Processor -- start 2024-04-25T00:11:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.018+0000 7fed42176740 1 -- start start 2024-04-25T00:11:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed42176740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652ae6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed42176740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 0x557652aeb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed42176740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 0x557652aede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed42176740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557652a49cb0 con 0x557652ae6840 2024-04-25T00:11:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed42176740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5576529f25b0 con 0x557652aeb9b0 2024-04-25T00:11:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed42176740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557652a45c70 con 0x557652ae5e50 2024-04-25T00:11:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed3d933640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652ae6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed2e7fc640 1 -- 172.21.15.110:0/1396405124 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fed28005220 con 0x7fed381006a0 2024-04-25T00:11:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed3d933640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652ae6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:54586/0 (socket says 172.21.15.110:54586) 2024-04-25T00:11:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed3d933640 1 -- 172.21.15.110:0/90101041 learned_addr learned my addr 172.21.15.110:0/90101041 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed3d933640 1 -- 172.21.15.110:0/90101041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 msgr2=0x557652aede40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:11:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed3d933640 1 --2- 172.21.15.110:0/90101041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 0x557652aede40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed3d933640 1 -- 172.21.15.110:0/90101041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 msgr2=0x557652aeb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed3d933640 1 --2- 172.21.15.110:0/90101041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 0x557652aeb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.019+0000 7fed3d933640 1 -- 172.21.15.110:0/90101041 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765275e210 con 0x557652ae5e50 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.020+0000 7fed3d933640 1 --2- 172.21.15.110:0/90101041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652ae6270 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fed30009f10 tx=0x7fed3000f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38e5e15c1838b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.020+0000 7fed2e7fc640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed240a2d30 0x7fed240a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.020+0000 7fed2e7fc640 1 -- 172.21.15.110:0/1396405124 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fed280ef1b0 con 0x7fed381006a0 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.020+0000 7fed237fe640 1 -- 172.21.15.110:0/90101041 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30003970 con 0x557652ae5e50 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.020+0000 7fed237fe640 1 -- 172.21.15.110:0/90101041 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed30004030 con 0x557652ae5e50 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.020+0000 7fed237fe640 1 -- 172.21.15.110:0/90101041 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3000c9d0 con 0x557652ae5e50 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.021+0000 7fed42176740 1 -- 172.21.15.110:0/90101041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 msgr2=0x557652ae6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.021+0000 7fed42176740 1 --2- 172.21.15.110:0/90101041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652ae6270 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fed30009f10 tx=0x7fed3000f7d0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed3d132640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed240a2d30 0x7fed240a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 -- 172.21.15.110:0/90101041 shutdown_connections 2024-04-25T00:11:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 --2- 172.21.15.110:0/90101041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 0x557652aede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 --2- 172.21.15.110:0/90101041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 0x557652aeb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 --2- 172.21.15.110:0/90101041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652ae6270 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 -- 172.21.15.110:0/90101041 >> 172.21.15.110:0/90101041 conn(0x557652ae14c0 msgr2=0x557652ae18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 -- 172.21.15.110:0/90101041 shutdown_connections 2024-04-25T00:11:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 -- 172.21.15.110:0/90101041 wait complete. 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 -- start start 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed42176740 1 Processor -- start 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.022+0000 7fed3d132640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed240a2d30 0x7fed240a51f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fed28009830 tx=0x7fed280a6040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed42176740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed42176740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed42176740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652aeb9b0 0x557652af78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed42176740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557652aee960 con 0x557652aeb9b0 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed42176740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557652a05b50 con 0x557652ae6840 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed42176740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557652aeeae0 con 0x557652ae5e50 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed42176740 10 client.0 _ll_register_callbacks cb 0x557652a40ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed3c931640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed3c931640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:60276/0 (socket says 172.21.15.110:60276) 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.023+0000 7fed3c931640 1 -- 172.21.15.110:0/723457795 learned_addr learned my addr 172.21.15.110:0/723457795 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.024+0000 7fed3d933640 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.024+0000 7fed3c931640 1 -- 172.21.15.110:0/723457795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 msgr2=0x557652af0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.024+0000 7fed3c931640 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.024+0000 7fed3c931640 1 -- 172.21.15.110:0/723457795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652aeb9b0 msgr2=0x557652af78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.024+0000 7fed3c931640 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652aeb9b0 0x557652af78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.024+0000 7fed3c931640 1 -- 172.21.15.110:0/723457795 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765282a620 con 0x557652ae6840 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.024+0000 7fed3c931640 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af0880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fed3400d670 tx=0x7fed3400db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca42a30690cbb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.025+0000 7fed20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.025+0000 7fed20ff9640 1 -- 172.21.15.110:0/723457795 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34005220 con 0x557652ae6840 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.025+0000 7fed20ff9640 1 -- 172.21.15.110:0/723457795 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fed340053c0 con 0x557652ae6840 2024-04-25T00:11:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.025+0000 7fed42176740 1 -- 172.21.15.110:0/723457795 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed30004030 con 0x557652ae6840 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.025+0000 7fed20ff9640 1 -- 172.21.15.110:0/723457795 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed340055c0 con 0x557652ae6840 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.025+0000 7fed42176740 10 client.25568 fetch_fsmap learned FSMap version 32 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.025+0000 7fed42176740 1 -- 172.21.15.110:0/723457795 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557652afaca0 con 0x557652ae6840 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed20ff9640 1 -- 172.21.15.110:0/723457795 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed30004030 con 0x557652ae6840 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed20ff9640 1 -- 172.21.15.110:0/723457795 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fed34004b10 con 0x557652ae6840 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed42176740 10 client.25568 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed42176740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed42176740 20 client.25568 populate_metadata read hostname 'smithi110' 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed42176740 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557652afc850 0x557652afec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed42176740 4 client.25568 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:11:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed42176740 1 -- 172.21.15.110:0/723457795 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557652a571f0 con 0x557652afc850 2024-04-25T00:11:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.026+0000 7fed3d933640 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557652afc850 0x557652afec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.027+0000 7fed20ff9640 10 client.25568 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.027+0000 7fed3d933640 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557652afc850 0x557652afec70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31f05887ecc7e5d6 server_cookie=8f2b74ff1f70ad1c in_seq=0 out_seq=0 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.028+0000 7fed20ff9640 1 -- 172.21.15.110:0/723457795 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557652a571f0 con 0x557652afc850 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.028+0000 7fed20ff9640 10 client.25568 handle_command_reply: tid=0 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.028+0000 7fed42176740 1 client.25568 shutdown 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 -- 172.21.15.110:0/723457795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 msgr2=0x557652af0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af0880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fed3400d670 tx=0x7fed3400db30 comp rx=0 tx=0).stop 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 -- 172.21.15.110:0/723457795 shutdown_connections 2024-04-25T00:11:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557652afc850 0x557652afec70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652aeb9b0 0x557652af78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af0880 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 --2- 172.21.15.110:0/723457795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 -- 172.21.15.110:0/723457795 >> 172.21.15.110:0/723457795 conn(0x557652ae14c0 msgr2=0x557652ae05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed20ff9640 0 client.25568 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 -- 172.21.15.110:0/723457795 shutdown_connections 2024-04-25T00:11:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 1 -- 172.21.15.110:0/723457795 wait complete. 2024-04-25T00:11:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.029+0000 7fed42176740 20 client.25568 trim_cache size 0 max 16384 2024-04-25T00:11:22.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:21 smithi110 ceph-mon[29187]: pgmap v630: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 MiB/s wr, 1.02k op/s 2024-04-25T00:11:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.054+0000 7fed42176740 1 Processor -- start 2024-04-25T00:11:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.054+0000 7fed42176740 1 -- start start 2024-04-25T00:11:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed42176740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae5e50 0x557652aef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed42176740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed42176740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652aeb9b0 0x557652af0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae5e50 0x557652aef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae5e50 0x557652aef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33026/0 (socket says 172.21.15.110:33026) 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 -- 172.21.15.110:0/3285387938 learned_addr learned my addr 172.21.15.110:0/3285387938 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed42176740 1 -- 172.21.15.110:0/3285387938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed3806e280 con 0x557652ae5e50 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed42176740 1 -- 172.21.15.110:0/3285387938 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed3806bb50 con 0x557652ae6840 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed42176740 1 -- 172.21.15.110:0/3285387938 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed381a29c0 con 0x557652aeb9b0 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3c931640 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652aeb9b0 0x557652af0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 -- 172.21.15.110:0/3285387938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652aeb9b0 msgr2=0x557652af0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652aeb9b0 0x557652af0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 -- 172.21.15.110:0/3285387938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 msgr2=0x557652af7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3d933640 1 -- 172.21.15.110:0/3285387938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557652aef420 con 0x557652ae5e50 2024-04-25T00:11:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.055+0000 7fed3c931640 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.056+0000 7fed3d933640 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae5e50 0x557652aef960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fed3000fcd0 tx=0x7fed30000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41faf0aa7bdae13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.056+0000 7fed22ffd640 1 -- 172.21.15.110:0/3285387938 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30026350 con 0x557652ae5e50 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.056+0000 7fed22ffd640 1 -- 172.21.15.110:0/3285387938 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed30003970 con 0x557652ae5e50 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.056+0000 7fed22ffd640 1 -- 172.21.15.110:0/3285387938 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3000c880 con 0x557652ae5e50 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 -- 172.21.15.110:0/3285387938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae5e50 msgr2=0x557652aef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae5e50 0x557652aef960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fed3000fcd0 tx=0x7fed30000f40 comp rx=0 tx=0).stop 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 -- 172.21.15.110:0/3285387938 shutdown_connections 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652aeb9b0 0x557652af0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652ae6840 0x557652af7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 --2- 172.21.15.110:0/3285387938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae5e50 0x557652aef960 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 -- 172.21.15.110:0/3285387938 >> 172.21.15.110:0/3285387938 conn(0x557652ae14c0 msgr2=0x557652ae18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 -- 172.21.15.110:0/3285387938 shutdown_connections 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.057+0000 7fed42176740 1 -- 172.21.15.110:0/3285387938 wait complete. 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.058+0000 7fed42176740 1 -- start start 2024-04-25T00:11:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.058+0000 7fed42176740 1 Processor -- start 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed42176740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed42176740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 0x557652af4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed42176740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 0x557652b60dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed42176740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed381a2fc0 con 0x557652ae6840 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed42176740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed3806b480 con 0x557652aeb9b0 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed42176740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed381a3690 con 0x557652ae5e50 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed42176740 10 client.0 _ll_register_callbacks cb 0x7fed34005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3c931640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 0x557652af4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3d933640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3c931640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 0x557652af4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33038/0 (socket says 172.21.15.110:33038) 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3c931640 1 -- 172.21.15.110:0/4116629967 learned_addr learned my addr 172.21.15.110:0/4116629967 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3d933640 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 0x557652b60dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3d933640 1 -- 172.21.15.110:0/4116629967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 msgr2=0x557652b60dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3d933640 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 0x557652b60dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3d933640 1 -- 172.21.15.110:0/4116629967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 msgr2=0x557652af4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3d933640 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 0x557652af4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3d933640 1 -- 172.21.15.110:0/4116629967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557652af7960 con 0x557652ae5e50 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.059+0000 7fed3d933640 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af4350 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d50 tx=0x7fed30004300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beaf14a4362ef5d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.060+0000 7fed20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.060+0000 7fed20ff9640 1 -- 172.21.15.110:0/4116629967 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30003b20 con 0x557652ae5e50 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.060+0000 7fed20ff9640 1 -- 172.21.15.110:0/4116629967 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fed30015070 con 0x557652ae5e50 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.060+0000 7fed42176740 1 -- 172.21.15.110:0/4116629967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed30003970 con 0x557652ae5e50 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.060+0000 7fed20ff9640 1 -- 172.21.15.110:0/4116629967 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed30003cc0 con 0x557652ae5e50 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.060+0000 7fed42176740 10 client.25555 fetch_fsmap learned FSMap version 32 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.060+0000 7fed42176740 1 -- 172.21.15.110:0/4116629967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557652b621c0 con 0x557652ae5e50 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed20ff9640 1 -- 172.21.15.110:0/4116629967 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed30003970 con 0x557652ae5e50 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed20ff9640 1 -- 172.21.15.110:0/4116629967 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fed3000b050 con 0x557652ae5e50 2024-04-25T00:11:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed42176740 10 client.25555 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:11:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed42176740 10 client.25555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:11:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed42176740 20 client.25555 populate_metadata read hostname 'smithi110' 2024-04-25T00:11:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed42176740 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557652b62970 0x557652b64d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed42176740 4 client.25555 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:11:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed42176740 1 -- 172.21.15.110:0/4116629967 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557652a51540 con 0x557652b62970 2024-04-25T00:11:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed3c931640 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557652b62970 0x557652b64d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed20ff9640 10 client.25555 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.061+0000 7fed3c931640 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557652b62970 0x557652b64d60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbd5c21aae2c868 server_cookie=cf6b6650ea6a6baa in_seq=0 out_seq=0 2024-04-25T00:11:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.075+0000 7fed20ff9640 1 -- 172.21.15.110:0/4116629967 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557652a51540 con 0x557652b62970 2024-04-25T00:11:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.075+0000 7fed20ff9640 10 client.25555 handle_command_reply: tid=0 2024-04-25T00:11:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.075+0000 7fed42176740 1 client.25555 shutdown 2024-04-25T00:11:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.075+0000 7fed42176740 1 -- 172.21.15.110:0/4116629967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 msgr2=0x557652af4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.075+0000 7fed42176740 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af4350 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d50 tx=0x7fed30004300 comp rx=0 tx=0).stop 2024-04-25T00:11:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.075+0000 7fed42176740 1 -- 172.21.15.110:0/4116629967 shutdown_connections 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed42176740 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557652b62970 0x557652b64d60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed42176740 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557652aeb9b0 0x557652b60dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed42176740 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557652ae6840 0x557652af4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed20ff9640 0 client.25555 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed42176740 1 --2- 172.21.15.110:0/4116629967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557652ae5e50 0x557652af4350 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed42176740 1 -- 172.21.15.110:0/4116629967 >> 172.21.15.110:0/4116629967 conn(0x557652ae14c0 msgr2=0x557652aef930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed42176740 1 -- 172.21.15.110:0/4116629967 shutdown_connections 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed42176740 1 -- 172.21.15.110:0/4116629967 wait complete. 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.076+0000 7fed42176740 20 client.25555 trim_cache size 0 max 16384 2024-04-25T00:11:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:11:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:11:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:11:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:11:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 -- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed240a2d30 msgr2=0x7fed240a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed240a2d30 0x7fed240a51f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fed28009830 tx=0x7fed280a6040 comp rx=0 tx=0).stop 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 -- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed381006a0 msgr2=0x7fed3810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed381006a0 0x7fed3810be40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fed28005e00 tx=0x7fed28004360 comp rx=0 tx=0).stop 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 -- 172.21.15.110:0/1396405124 shutdown_connections 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fed240a2d30 0x7fed240a51f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed38102250 0x7fed3810a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed38101890 0x7fed3810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 --2- 172.21.15.110:0/1396405124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed381006a0 0x7fed3810be40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 -- 172.21.15.110:0/1396405124 >> 172.21.15.110:0/1396405124 conn(0x7fed380fbc10 msgr2=0x7fed38104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 -- 172.21.15.110:0/1396405124 shutdown_connections 2024-04-25T00:11:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:22.077+0000 7fed3f396640 1 -- 172.21.15.110:0/1396405124 wait complete. 2024-04-25T00:11:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:21 smithi173 ceph-mon[32402]: pgmap v630: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 MiB/s wr, 1.02k op/s 2024-04-25T00:11:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:23 smithi188 ceph-mon[33285]: pgmap v631: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 1.07k op/s 2024-04-25T00:11:24.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:23 smithi110 ceph-mon[29187]: pgmap v631: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 1.07k op/s 2024-04-25T00:11:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:23 smithi173 ceph-mon[32402]: pgmap v631: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.0 MiB/s wr, 1.07k op/s 2024-04-25T00:11:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:25 smithi188 ceph-mon[33285]: pgmap v632: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 1.04k op/s 2024-04-25T00:11:26.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:25 smithi110 ceph-mon[29187]: pgmap v632: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 1.04k op/s 2024-04-25T00:11:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:25 smithi173 ceph-mon[32402]: pgmap v632: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 1.04k op/s 2024-04-25T00:11:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 500.0 s, 796.4 tps, lat 125.767 ms stddev 22.827 2024-04-25T00:11:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:27 smithi188 ceph-mon[33285]: pgmap v633: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 MiB/s wr, 870 op/s 2024-04-25T00:11:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:27 smithi110 ceph-mon[29187]: pgmap v633: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 MiB/s wr, 870 op/s 2024-04-25T00:11:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:27 smithi173 ceph-mon[32402]: pgmap v633: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 MiB/s wr, 870 op/s 2024-04-25T00:11:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:29 smithi188 ceph-mon[33285]: pgmap v634: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.2 MiB/s wr, 1.06k op/s 2024-04-25T00:11:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:29 smithi110 ceph-mon[29187]: pgmap v634: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.2 MiB/s wr, 1.06k op/s 2024-04-25T00:11:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:29 smithi173 ceph-mon[32402]: pgmap v634: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.2 MiB/s wr, 1.06k op/s 2024-04-25T00:11:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 505.0 s, 834.0 tps, lat 119.889 ms stddev 17.679 2024-04-25T00:11:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:31 smithi188 ceph-mon[33285]: pgmap v635: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 926 op/s 2024-04-25T00:11:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:31 smithi110 ceph-mon[29187]: pgmap v635: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 926 op/s 2024-04-25T00:11:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:31 smithi173 ceph-mon[32402]: pgmap v635: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 926 op/s 2024-04-25T00:11:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:33 smithi188 ceph-mon[33285]: pgmap v636: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 991 op/s 2024-04-25T00:11:34.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:33 smithi110 ceph-mon[29187]: pgmap v636: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 991 op/s 2024-04-25T00:11:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:33 smithi173 ceph-mon[32402]: pgmap v636: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.7 MiB/s wr, 991 op/s 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.205+0000 7f2ed245d640 1 -- 172.21.15.110:0/2924795754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 msgr2=0x7f2ecc073b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.205+0000 7f2ed245d640 1 --2- 172.21.15.110:0/2924795754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 0x7f2ecc073b50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009900 tx=0x7f2ebc02a480 comp rx=0 tx=0).stop 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.205+0000 7f2ed245d640 1 -- 172.21.15.110:0/2924795754 shutdown_connections 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.205+0000 7f2ed245d640 1 --2- 172.21.15.110:0/2924795754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ecc109680 0x7f2ecc10bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.205+0000 7f2ed245d640 1 --2- 172.21.15.110:0/2924795754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ecc074120 0x7f2ecc074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.205+0000 7f2ed245d640 1 --2- 172.21.15.110:0/2924795754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 0x7f2ecc073b50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.205+0000 7f2ed245d640 1 -- 172.21.15.110:0/2924795754 >> 172.21.15.110:0/2924795754 conn(0x7f2ecc0fcd60 msgr2=0x7f2ecc0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.206+0000 7f2ed245d640 1 -- 172.21.15.110:0/2924795754 shutdown_connections 2024-04-25T00:11:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.206+0000 7f2ed245d640 1 -- 172.21.15.110:0/2924795754 wait complete. 2024-04-25T00:11:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.206+0000 7f2ed245d640 1 Processor -- start 2024-04-25T00:11:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.206+0000 7f2ed245d640 1 -- start start 2024-04-25T00:11:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ed245d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ecc074120 0x7f2ecc199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ed245d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 0x7f2ecc19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ed245d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ecc109680 0x7f2ecc1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 0x7f2ecc19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ed245d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc10f530 con 0x7f2ecc075700 2024-04-25T00:11:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ed245d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc10f3b0 con 0x7f2ecc109680 2024-04-25T00:11:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ed245d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc10f6b0 con 0x7f2ecc074120 2024-04-25T00:11:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 0x7f2ecc19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51420/0 (socket says 172.21.15.110:51420) 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/3697441375 learned_addr learned my addr 172.21.15.110:0/3697441375 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ecc074120 0x7f2ecc199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ed09fa640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ecc109680 0x7f2ecc1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/3697441375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ecc074120 msgr2=0x7f2ecc199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ecc074120 0x7f2ecc199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/3697441375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ecc109680 msgr2=0x7f2ecc1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ecc109680 0x7f2ecc1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.207+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/3697441375 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc1a1750 con 0x7f2ecc075700 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.208+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 0x7f2ecc19a070 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009d50 tx=0x7f2eb400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.208+0000 7f2ec97fa640 1 -- 172.21.15.110:0/3697441375 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4003b70 con 0x7f2ecc075700 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.208+0000 7f2ed245d640 1 -- 172.21.15.110:0/3697441375 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ecc1a1a40 con 0x7f2ecc075700 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.208+0000 7f2ed245d640 1 -- 172.21.15.110:0/3697441375 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ecc1a1fe0 con 0x7f2ecc075700 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.208+0000 7f2ec97fa640 1 -- 172.21.15.110:0/3697441375 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2eb4003d10 con 0x7f2ecc075700 2024-04-25T00:11:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.208+0000 7f2ec97fa640 1 -- 172.21.15.110:0/3697441375 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4004d20 con 0x7f2ecc075700 2024-04-25T00:11:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.211+0000 7f2ec97fa640 1 -- 172.21.15.110:0/3697441375 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2eb40045d0 con 0x7f2ecc075700 2024-04-25T00:11:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed516a740 1 Processor -- start 2024-04-25T00:11:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ec97fa640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f2eac0a2c30 0x7f2eac0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed516a740 1 -- start start 2024-04-25T00:11:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b36005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed09fa640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f2eac0a2c30 0x7f2eac0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ec97fa640 1 -- 172.21.15.110:0/3697441375 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2eb40cf6a0 con 0x7f2ecc075700 2024-04-25T00:11:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b3600a9b0 0x558b3600ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed516a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558b35f68cb0 con 0x558b36005840 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed516a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558b35f115b0 con 0x558b3600a9b0 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ed516a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b35f64c70 con 0x558b36004e50 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51422/0 (socket says 172.21.15.110:51422) 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b36005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.212+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/2649995088 learned_addr learned my addr 172.21.15.110:0/2649995088 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ed09fa640 1 --2- 172.21.15.110:0/2649995088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b3600a9b0 0x558b3600ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ecbfff640 1 -- 172.21.15.110:0/2649995088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b3600a9b0 msgr2=0x558b3600ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/2649995088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b3600a9b0 0x558b3600ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ecbfff640 1 -- 172.21.15.110:0/2649995088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 msgr2=0x558b3600a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/2649995088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ecbfff640 1 -- 172.21.15.110:0/2649995088 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b35c7d210 con 0x558b36004e50 2024-04-25T00:11:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ed09fa640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f2eac0a2c30 0x7f2eac0a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4011440 tx=0x7f2ec4011840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/2649995088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b36005270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc02a9a0 tx=0x7f2ebc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cec941d4c450ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ec27fc640 1 -- 172.21.15.110:0/2649995088 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc035030 con 0x558b36004e50 2024-04-25T00:11:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ec27fc640 1 -- 172.21.15.110:0/2649995088 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ebc040070 con 0x558b36004e50 2024-04-25T00:11:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.213+0000 7f2ec27fc640 1 -- 172.21.15.110:0/2649995088 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0318f0 con 0x558b36004e50 2024-04-25T00:11:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.214+0000 7f2ed516a740 1 -- 172.21.15.110:0/2649995088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 msgr2=0x558b36005270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.214+0000 7f2ed516a740 1 --2- 172.21.15.110:0/2649995088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b36005270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc02a9a0 tx=0x7f2ebc005c00 comp rx=0 tx=0).stop 2024-04-25T00:11:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.214+0000 7f2ed516a740 1 -- 172.21.15.110:0/2649995088 shutdown_connections 2024-04-25T00:11:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.214+0000 7f2ed516a740 1 --2- 172.21.15.110:0/2649995088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b3600a9b0 0x558b3600ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.214+0000 7f2ed516a740 1 --2- 172.21.15.110:0/2649995088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.214+0000 7f2ed516a740 1 --2- 172.21.15.110:0/2649995088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b36005270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.214+0000 7f2ed516a740 1 -- 172.21.15.110:0/2649995088 >> 172.21.15.110:0/2649995088 conn(0x558b360004c0 msgr2=0x558b360008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.215+0000 7f2ed516a740 1 -- 172.21.15.110:0/2649995088 shutdown_connections 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.215+0000 7f2ed516a740 1 -- 172.21.15.110:0/2649995088 wait complete. 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.215+0000 7f2ed516a740 1 -- start start 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.215+0000 7f2ed516a740 1 Processor -- start 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 0x558b360167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:58650/0 (socket says 172.21.15.110:58650) 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed516a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558b3600d900 con 0x558b3600a9b0 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed516a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558b35f24b50 con 0x558b36004e50 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed516a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b3600da80 con 0x558b36005840 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:47672/0 (socket says 172.21.15.110:47672) 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/4282636822 learned_addr learned my addr 172.21.15.110:0/4282636822 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed09fa640 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 0x558b360167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/4282636822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 msgr2=0x558b3600f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/4282636822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 msgr2=0x558b360167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 0x558b360167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ecb7fe640 1 -- 172.21.15.110:0/4282636822 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b35d49620 con 0x558b36005840 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed09fa640 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 0x558b360167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.216+0000 7f2ed516a740 10 client.0 _ll_register_callbacks cb 0x558b35f5fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.217+0000 7f2ed516a740 1 -- 172.21.15.110:0/4282636822 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ebc040070 con 0x558b36005840 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.217+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36005840 0x558b3600f7b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2eb409bc30 tx=0x7f2eb409b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8629134eccc31559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.217+0000 7f2ebb7fe640 10 client.25567 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.217+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4282636822 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40b4070 con 0x558b36005840 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.217+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4282636822 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2eb40bab70 con 0x558b36005840 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.217+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4282636822 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2eb40bad70 con 0x558b36005840 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.217+0000 7f2ed516a740 10 client.25567 fetch_fsmap learned FSMap version 32 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.217+0000 7f2ed516a740 1 -- 172.21.15.110:0/4282636822 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=32}) -- 0x558b36019ad0 con 0x558b36005840 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.218+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4282636822 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f2eb409b380 con 0x558b36005840 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.218+0000 7f2ed516a740 10 client.25567 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.218+0000 7f2ed516a740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.218+0000 7f2ed516a740 20 client.25567 populate_metadata read hostname 'smithi110' 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.218+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558b3601b100 0x558b3601d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.218+0000 7f2ed516a740 4 client.25567 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.218+0000 7f2ed516a740 1 -- 172.21.15.110:0/4282636822 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b35f761f0 con 0x558b3601b100 2024-04-25T00:11:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.218+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558b3601b100 0x558b3601d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.219+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558b3601b100 0x558b3601d570 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182c845c4499f2b0 server_cookie=beffb74610b6defd in_seq=0 out_seq=0 2024-04-25T00:11:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.219+0000 7f2ebb7fe640 10 client.25567 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.219+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4282636822 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b35f761f0 con 0x558b3601b100 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.219+0000 7f2ebb7fe640 10 client.25567 handle_command_reply: tid=0 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.219+0000 7f2ed516a740 1 client.25567 shutdown 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 -- 172.21.15.110:0/4282636822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36005840 msgr2=0x558b3600f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36005840 0x558b3600f7b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2eb409bc30 tx=0x7f2eb409b890 comp rx=0 tx=0).stop 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 -- 172.21.15.110:0/4282636822 shutdown_connections 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558b3601b100 0x558b3601d570 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 0x558b360167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4282636822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 -- 172.21.15.110:0/4282636822 >> 172.21.15.110:0/4282636822 conn(0x558b360004c0 msgr2=0x558b35fff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ebb7fe640 0 client.25567 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 -- 172.21.15.110:0/4282636822 shutdown_connections 2024-04-25T00:11:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.221+0000 7f2ed516a740 1 -- 172.21.15.110:0/4282636822 wait complete. 2024-04-25T00:11:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.222+0000 7f2ed516a740 20 client.25567 trim_cache size 0 max 16384 2024-04-25T00:11:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.247+0000 7f2ed516a740 1 Processor -- start 2024-04-25T00:11:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.247+0000 7f2ed516a740 1 -- start start 2024-04-25T00:11:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3600a9b0 0x558b360167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:58664/0 (socket says 172.21.15.110:58664) 2024-04-25T00:11:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ed516a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1ad660 con 0x558b36005840 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecbfff640 1 -- 172.21.15.110:0/784911608 learned_addr learned my addr 172.21.15.110:0/784911608 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ed516a740 1 -- 172.21.15.110:0/784911608 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1aaf30 con 0x558b36004e50 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ed516a740 1 -- 172.21.15.110:0/784911608 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1a8d80 con 0x558b3600a9b0 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ed09fa640 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3600a9b0 0x558b360167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecbfff640 1 -- 172.21.15.110:0/784911608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3600a9b0 msgr2=0x558b360167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3600a9b0 0x558b360167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecbfff640 1 -- 172.21.15.110:0/784911608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 msgr2=0x558b3600f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecbfff640 1 -- 172.21.15.110:0/784911608 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b360055e0 con 0x558b36004e50 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.248+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.249+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009900 tx=0x7f2ebc004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6f7a2fd5c0042c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.249+0000 7f2ec1ffb640 1 -- 172.21.15.110:0/784911608 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc035040 con 0x558b36004e50 2024-04-25T00:11:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.249+0000 7f2ec1ffb640 1 -- 172.21.15.110:0/784911608 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ebc057070 con 0x558b36004e50 2024-04-25T00:11:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.249+0000 7f2ec1ffb640 1 -- 172.21.15.110:0/784911608 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc031900 con 0x558b36004e50 2024-04-25T00:11:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.250+0000 7f2ed516a740 1 -- 172.21.15.110:0/784911608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 msgr2=0x558b3600f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.250+0000 7f2ed516a740 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009900 tx=0x7f2ebc004250 comp rx=0 tx=0).stop 2024-04-25T00:11:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.251+0000 7f2ed516a740 1 -- 172.21.15.110:0/784911608 shutdown_connections 2024-04-25T00:11:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.251+0000 7f2ed516a740 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b3600a9b0 0x558b360167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.251+0000 7f2ed516a740 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b36005840 0x558b3600f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.251+0000 7f2ed516a740 1 --2- 172.21.15.110:0/784911608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36004e50 0x558b3600f270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.251+0000 7f2ed516a740 1 -- 172.21.15.110:0/784911608 >> 172.21.15.110:0/784911608 conn(0x558b360004c0 msgr2=0x558b3600de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.251+0000 7f2ed516a740 1 -- 172.21.15.110:0/784911608 shutdown_connections 2024-04-25T00:11:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.251+0000 7f2ed516a740 1 -- 172.21.15.110:0/784911608 wait complete. 2024-04-25T00:11:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.252+0000 7f2ed516a740 1 -- start start 2024-04-25T00:11:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.252+0000 7f2ed516a740 1 Processor -- start 2024-04-25T00:11:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.253+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b360165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.253+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36005840 0x558b36013060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.253+0000 7f2ed516a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 0x558b360135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.253+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b360165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.253+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b360165c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:47682/0 (socket says 172.21.15.110:47682) 2024-04-25T00:11:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.253+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36005840 0x558b36013060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.253+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36005840 0x558b36013060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:58678/0 (socket says 172.21.15.110:58678) 2024-04-25T00:11:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.253+0000 7f2ecbfff640 1 -- 172.21.15.110:0/4083243457 learned_addr learned my addr 172.21.15.110:0/4083243457 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1a93f0 con 0x558b3600a9b0 2024-04-25T00:11:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1aa860 con 0x558b36005840 2024-04-25T00:11:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1a9ac0 con 0x558b36004e50 2024-04-25T00:11:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ed516a740 10 client.0 _ll_register_callbacks cb 0x7f2eb40098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ecbfff640 1 -- 172.21.15.110:0/4083243457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36005840 msgr2=0x558b36013060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36005840 0x558b36013060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ecbfff640 1 -- 172.21.15.110:0/4083243457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 msgr2=0x558b360135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 0x558b360135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ecbfff640 1 -- 172.21.15.110:0/4083243457 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b36013be0 con 0x558b36004e50 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ecbfff640 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b360165c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006010 tx=0x7f2ebc0045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=389a5c51d9dd6310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.254+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ebc057070 con 0x558b36004e50 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.255+0000 7f2ebb7fe640 10 client.25573 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:11:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.255+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4083243457 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc035070 con 0x558b36004e50 2024-04-25T00:11:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.255+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4083243457 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f2ebc031c90 con 0x558b36004e50 2024-04-25T00:11:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.256+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4083243457 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2ebc0035f0 con 0x558b36004e50 2024-04-25T00:11:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.256+0000 7f2ed516a740 10 client.25573 fetch_fsmap learned FSMap version 32 2024-04-25T00:11:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.256+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=32}) -- 0x558b36088e70 con 0x558b36004e50 2024-04-25T00:11:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.257+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4083243457 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f2ebc003790 con 0x558b36004e50 2024-04-25T00:11:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.257+0000 7f2ed516a740 10 client.25573 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:11:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.257+0000 7f2ed516a740 10 client.25573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:11:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.257+0000 7f2ed516a740 20 client.25573 populate_metadata read hostname 'smithi110' 2024-04-25T00:11:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.257+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558b36089250 0x558b3608b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.257+0000 7f2ed516a740 4 client.25573 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:11:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.257+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2ebc057070 con 0x558b36089250 2024-04-25T00:11:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.257+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558b36089250 0x558b3608b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.258+0000 7f2ecb7fe640 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558b36089250 0x558b3608b640 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea17e619e5ab93c6 server_cookie=8ee5de686f4421d4 in_seq=0 out_seq=0 2024-04-25T00:11:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.258+0000 7f2ebb7fe640 10 client.25573 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.278+0000 7f2ebb7fe640 1 -- 172.21.15.110:0/4083243457 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2ebc057070 con 0x558b36089250 2024-04-25T00:11:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ebb7fe640 10 client.25573 handle_command_reply: tid=0 2024-04-25T00:11:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 client.25573 shutdown 2024-04-25T00:11:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 msgr2=0x558b360165c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b360165c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006010 tx=0x7f2ebc0045f0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 shutdown_connections 2024-04-25T00:11:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558b36089250 0x558b3608b640 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b3600a9b0 0x558b360135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b36005840 0x558b36013060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 --2- 172.21.15.110:0/4083243457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b36004e50 0x558b360165c0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006010 tx=0x7f2ebc0045f0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 >> 172.21.15.110:0/4083243457 conn(0x558b360004c0 msgr2=0x558b3600f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.279+0000 7f2ebb7fe640 0 client.25573 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.281+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 shutdown_connections 2024-04-25T00:11:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.282+0000 7f2ed516a740 1 -- 172.21.15.110:0/4083243457 wait complete. 2024-04-25T00:11:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.283+0000 7f2ed516a740 20 client.25573 trim_cache size 0 max 16384 2024-04-25T00:11:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:11:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:11:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:11:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:11:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.284+0000 7f2ed245d640 1 -- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f2eac0a2c30 msgr2=0x7f2eac0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.284+0000 7f2ed245d640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f2eac0a2c30 0x7f2eac0a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4011440 tx=0x7f2ec4011840 comp rx=0 tx=0).stop 2024-04-25T00:11:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.284+0000 7f2ed245d640 1 -- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 msgr2=0x7f2ecc19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.284+0000 7f2ed245d640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 0x7f2ecc19a070 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009d50 tx=0x7f2eb400c6a0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.285+0000 7f2ed245d640 1 -- 172.21.15.110:0/3697441375 shutdown_connections 2024-04-25T00:11:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.285+0000 7f2ed245d640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f2eac0a2c30 0x7f2eac0a50f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.285+0000 7f2ed245d640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ecc109680 0x7f2ecc1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.285+0000 7f2ed245d640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ecc075700 0x7f2ecc19a070 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.285+0000 7f2ed245d640 1 --2- 172.21.15.110:0/3697441375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ecc074120 0x7f2ecc199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.285+0000 7f2ed245d640 1 -- 172.21.15.110:0/3697441375 >> 172.21.15.110:0/3697441375 conn(0x7f2ecc0fcd60 msgr2=0x7f2ecc105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.285+0000 7f2ed245d640 1 -- 172.21.15.110:0/3697441375 shutdown_connections 2024-04-25T00:11:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:35.285+0000 7f2ed245d640 1 -- 172.21.15.110:0/3697441375 wait complete. 2024-04-25T00:11:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:35 smithi110 ceph-mon[29187]: pgmap v637: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.9 MiB/s wr, 891 op/s 2024-04-25T00:11:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:35 smithi173 ceph-mon[32402]: pgmap v637: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.9 MiB/s wr, 891 op/s 2024-04-25T00:11:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:35 smithi188 ceph-mon[33285]: pgmap v637: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.9 MiB/s wr, 891 op/s 2024-04-25T00:11:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 510.0 s, 821.4 tps, lat 121.815 ms stddev 23.504 2024-04-25T00:11:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:37 smithi110 ceph-mon[29187]: pgmap v638: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 751 op/s 2024-04-25T00:11:38.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:37 smithi188 ceph-mon[33285]: pgmap v638: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 751 op/s 2024-04-25T00:11:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:37 smithi173 ceph-mon[32402]: pgmap v638: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 751 op/s 2024-04-25T00:11:39.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:38 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:39.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:38 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:38 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:40.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:39 smithi110 ceph-mon[29187]: pgmap v639: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.8 MiB/s wr, 999 op/s 2024-04-25T00:11:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:39 smithi173 ceph-mon[32402]: pgmap v639: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.8 MiB/s wr, 999 op/s 2024-04-25T00:11:40.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:39 smithi188 ceph-mon[33285]: pgmap v639: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.8 MiB/s wr, 999 op/s 2024-04-25T00:11:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 515.0 s, 826.6 tps, lat 120.625 ms stddev 16.801 2024-04-25T00:11:42.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:41 smithi110 ceph-mon[29187]: pgmap v640: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 849 op/s 2024-04-25T00:11:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:41 smithi173 ceph-mon[32402]: pgmap v640: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 849 op/s 2024-04-25T00:11:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:41 smithi188 ceph-mon[33285]: pgmap v640: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 849 op/s 2024-04-25T00:11:43.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:42 smithi173 ceph-mon[32402]: pgmap v641: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-04-25T00:11:43.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:42 smithi188 ceph-mon[33285]: pgmap v641: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-04-25T00:11:43.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:42 smithi110 ceph-mon[29187]: pgmap v641: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-04-25T00:11:44.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:44.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:44.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:11:44.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:44.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:44.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:11:44.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:44.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:44.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:11:45.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:45 smithi188 ceph-mon[33285]: pgmap v642: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 853 op/s 2024-04-25T00:11:45.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:45 smithi110 ceph-mon[29187]: pgmap v642: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 853 op/s 2024-04-25T00:11:45.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:45 smithi173 ceph-mon[32402]: pgmap v642: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 853 op/s 2024-04-25T00:11:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 520.0 s, 816.0 tps, lat 122.547 ms stddev 22.206 2024-04-25T00:11:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:47 smithi188 ceph-mon[33285]: pgmap v643: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-04-25T00:11:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:47 smithi110 ceph-mon[29187]: pgmap v643: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-04-25T00:11:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:47 smithi173 ceph-mon[32402]: pgmap v643: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-04-25T00:11:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 -- 172.21.15.110:0/2635099900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 msgr2=0x7f76e810baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 --2- 172.21.15.110:0/2635099900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 0x7f76e810baf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009ca0 tx=0x7f76e002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 -- 172.21.15.110:0/2635099900 shutdown_connections 2024-04-25T00:11:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 --2- 172.21.15.110:0/2635099900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 0x7f76e810baf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 --2- 172.21.15.110:0/2635099900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e80741a0 0x7f76e8074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 --2- 172.21.15.110:0/2635099900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8075780 0x7f76e8073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 -- 172.21.15.110:0/2635099900 >> 172.21.15.110:0/2635099900 conn(0x7f76e80fcd30 msgr2=0x7f76e80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 -- 172.21.15.110:0/2635099900 shutdown_connections 2024-04-25T00:11:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.431+0000 7f76ee85d640 1 -- 172.21.15.110:0/2635099900 wait complete. 2024-04-25T00:11:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.432+0000 7f76ee85d640 1 Processor -- start 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.432+0000 7f76ee85d640 1 -- start start 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.432+0000 7f76ee85d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e80741a0 0x7f76e8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.432+0000 7f76ee85d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8075780 0x7f76e819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.432+0000 7f76ee85d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 0x7f76e81a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76ee85d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810f510 con 0x7f76e8075780 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76ee85d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810f390 con 0x7f76e80741a0 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76ee85d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76e810f690 con 0x7f76e8109660 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8075780 0x7f76e819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76ecdfa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 0x7f76e81a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e80741a0 0x7f76e8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8075780 0x7f76e819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57880/0 (socket says 172.21.15.110:57880) 2024-04-25T00:11:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e77fe640 1 -- 172.21.15.110:0/4151469332 learned_addr learned my addr 172.21.15.110:0/4151469332 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e80741a0 0x7f76e8199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:59218/0 (socket says 172.21.15.110:59218) 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e77fe640 1 -- 172.21.15.110:0/4151469332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 msgr2=0x7f76e81a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e77fe640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 0x7f76e81a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e77fe640 1 -- 172.21.15.110:0/4151469332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e80741a0 msgr2=0x7f76e8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e77fe640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e80741a0 0x7f76e8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76e77fe640 1 -- 172.21.15.110:0/4151469332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81a1720 con 0x7f76e8075780 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.433+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 0x7f76e81a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.434+0000 7f76e77fe640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8075780 0x7f76e819a040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00b4a0 tx=0x7f76dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.434+0000 7f76e57fa640 1 -- 172.21.15.110:0/4151469332 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc014070 con 0x7f76e8075780 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.434+0000 7f76e57fa640 1 -- 172.21.15.110:0/4151469332 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76dc010ac0 con 0x7f76e8075780 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.434+0000 7f76ee85d640 1 -- 172.21.15.110:0/4151469332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81a1a10 con 0x7f76e8075780 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.434+0000 7f76e57fa640 1 -- 172.21.15.110:0/4151469332 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc017c30 con 0x7f76e8075780 2024-04-25T00:11:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.434+0000 7f76ee85d640 1 -- 172.21.15.110:0/4151469332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a1f50 con 0x7f76e8075780 2024-04-25T00:11:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.437+0000 7f76e57fa640 1 -- 172.21.15.110:0/4151469332 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f76dc017070 con 0x7f76e8075780 2024-04-25T00:11:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.437+0000 7f76f156d740 1 Processor -- start 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.437+0000 7f76f156d740 1 -- start start 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76f156d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x56133411c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76f156d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x561334121340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76f156d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x561334123e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76f156d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56133407fcb0 con 0x5613341219b0 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76f156d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5613340285b0 con 0x56133411be50 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76f156d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56133407bc70 con 0x56133411c840 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x561334121340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x561334123e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76e57fa640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76d00a2d30 0x7f76d00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x561334121340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:57202/0 (socket says 172.21.15.110:57202) 2024-04-25T00:11:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76e7fff640 1 -- 172.21.15.110:0/495135300 learned_addr learned my addr 172.21.15.110:0/495135300 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/495135300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x56133411c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76e57fa640 1 -- 172.21.15.110:0/4151469332 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f76dc0ce5f0 con 0x7f76e8075780 2024-04-25T00:11:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76ecdfa640 1 -- 172.21.15.110:0/495135300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 msgr2=0x561334121340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.438+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/495135300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x561334121340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.439+0000 7f76ecdfa640 1 -- 172.21.15.110:0/495135300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 msgr2=0x561334123e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.439+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/495135300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x561334123e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.439+0000 7f76ecdfa640 1 -- 172.21.15.110:0/495135300 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561333d94210 con 0x56133411be50 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.439+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76d00a2d30 0x7f76d00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.439+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/495135300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x56133411c270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f76e0031040 tx=0x7f76e0009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0d27792d01036f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.439+0000 7f76ce7fc640 1 -- 172.21.15.110:0/495135300 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0003970 con 0x56133411be50 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.439+0000 7f76ce7fc640 1 -- 172.21.15.110:0/495135300 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e0003d10 con 0x56133411be50 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.440+0000 7f76ce7fc640 1 -- 172.21.15.110:0/495135300 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e00309a0 con 0x56133411be50 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.440+0000 7f76f156d740 1 -- 172.21.15.110:0/495135300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 msgr2=0x56133411c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.440+0000 7f76f156d740 1 --2- 172.21.15.110:0/495135300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x56133411c270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f76e0031040 tx=0x7f76e0009a20 comp rx=0 tx=0).stop 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.440+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76d00a2d30 0x7f76d00a51f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f76e0006010 tx=0x7f76e0048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.441+0000 7f76f156d740 1 -- 172.21.15.110:0/495135300 shutdown_connections 2024-04-25T00:11:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.441+0000 7f76f156d740 1 --2- 172.21.15.110:0/495135300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x561334123e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.441+0000 7f76f156d740 1 --2- 172.21.15.110:0/495135300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x561334121340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.441+0000 7f76f156d740 1 --2- 172.21.15.110:0/495135300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x56133411c270 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.441+0000 7f76f156d740 1 -- 172.21.15.110:0/495135300 >> 172.21.15.110:0/495135300 conn(0x5613341174c0 msgr2=0x5613341178c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.441+0000 7f76f156d740 1 -- 172.21.15.110:0/495135300 shutdown_connections 2024-04-25T00:11:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.441+0000 7f76f156d740 1 -- 172.21.15.110:0/495135300 wait complete. 2024-04-25T00:11:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.441+0000 7f76f156d740 1 -- start start 2024-04-25T00:11:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76f156d740 1 Processor -- start 2024-04-25T00:11:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76f156d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x5613341261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76f156d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x5613341266e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76f156d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x56133412d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76f156d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561334124830 con 0x5613341219b0 2024-04-25T00:11:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76f156d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56133403bb50 con 0x56133411be50 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76f156d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5613341249b0 con 0x56133411c840 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76f156d740 10 client.0 _ll_register_callbacks cb 0x561334076ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x5613341266e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x5613341266e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:57210/0 (socket says 172.21.15.110:57210) 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e77fe640 1 -- 172.21.15.110:0/758067111 learned_addr learned my addr 172.21.15.110:0/758067111 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e7fff640 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x5613341261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x56133412d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e77fe640 1 -- 172.21.15.110:0/758067111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 msgr2=0x5613341261a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e77fe640 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x5613341261a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e77fe640 1 -- 172.21.15.110:0/758067111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 msgr2=0x56133412d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e77fe640 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x56133412d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.442+0000 7f76e77fe640 1 -- 172.21.15.110:0/758067111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561333e60620 con 0x56133411c840 2024-04-25T00:11:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76e77fe640 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x5613341266e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00b470 tx=0x7f76dc09a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff7e73fa8feacfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76c77fe640 1 -- 172.21.15.110:0/758067111 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc01e030 con 0x56133411c840 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76c77fe640 1 -- 172.21.15.110:0/758067111 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f76dc0991d0 con 0x56133411c840 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76f156d740 1 -- 172.21.15.110:0/758067111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e0003d10 con 0x56133411c840 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76c77fe640 1 -- 172.21.15.110:0/758067111 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f76dc0a3a90 con 0x56133411c840 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76f156d740 10 client.25585 fetch_fsmap learned FSMap version 32 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76f156d740 1 -- 172.21.15.110:0/758067111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561334130a00 con 0x56133411c840 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76c77fe640 1 -- 172.21.15.110:0/758067111 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e0003d10 con 0x56133411c840 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76c77fe640 1 -- 172.21.15.110:0/758067111 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f76dc0a3600 con 0x56133411c840 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76f156d740 10 client.25585 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76f156d740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76f156d740 20 client.25585 populate_metadata read hostname 'smithi110' 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.443+0000 7f76f156d740 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x561334132080 0x5613341344f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.444+0000 7f76f156d740 4 client.25585 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:11:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.444+0000 7f76f156d740 1 -- 172.21.15.110:0/758067111 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56133408d1f0 con 0x561334132080 2024-04-25T00:11:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.444+0000 7f76e7fff640 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x561334132080 0x5613341344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.444+0000 7f76c77fe640 10 client.25585 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.444+0000 7f76e7fff640 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x561334132080 0x5613341344f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a75da21f4b0d09f server_cookie=bd778df37235726c in_seq=0 out_seq=0 2024-04-25T00:11:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.445+0000 7f76c77fe640 1 -- 172.21.15.110:0/758067111 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56133408d1f0 con 0x561334132080 2024-04-25T00:11:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.445+0000 7f76c77fe640 10 client.25585 handle_command_reply: tid=0 2024-04-25T00:11:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.445+0000 7f76f156d740 1 client.25585 shutdown 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.445+0000 7f76f156d740 1 -- 172.21.15.110:0/758067111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 msgr2=0x5613341266e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.445+0000 7f76f156d740 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x5613341266e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00b470 tx=0x7f76dc09a8f0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.445+0000 7f76f156d740 1 -- 172.21.15.110:0/758067111 shutdown_connections 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.445+0000 7f76f156d740 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x561334132080 0x5613341344f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.446+0000 7f76c77fe640 0 client.25585 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.446+0000 7f76f156d740 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x56133412d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.446+0000 7f76f156d740 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411c840 0x5613341266e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.446+0000 7f76f156d740 1 --2- 172.21.15.110:0/758067111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411be50 0x5613341261a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.446+0000 7f76f156d740 1 -- 172.21.15.110:0/758067111 >> 172.21.15.110:0/758067111 conn(0x5613341174c0 msgr2=0x5613341165c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.446+0000 7f76f156d740 1 -- 172.21.15.110:0/758067111 shutdown_connections 2024-04-25T00:11:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.446+0000 7f76f156d740 1 -- 172.21.15.110:0/758067111 wait complete. 2024-04-25T00:11:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.446+0000 7f76f156d740 20 client.25585 trim_cache size 0 max 16384 2024-04-25T00:11:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.467+0000 7f76f156d740 1 Processor -- start 2024-04-25T00:11:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.467+0000 7f76f156d740 1 -- start start 2024-04-25T00:11:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76f156d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x5613341257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76f156d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411c840 0x561334124d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x5613341257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x5613341257c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:57212/0 (socket says 172.21.15.110:57212) 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76e7fff640 1 -- 172.21.15.110:0/2518615784 learned_addr learned my addr 172.21.15.110:0/2518615784 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76f156d740 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x5613341178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76ecdfa640 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x5613341178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76e77fe640 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411c840 0x561334124d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76f156d740 1 -- 172.21.15.110:0/2518615784 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81a90f0 con 0x5613341219b0 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76f156d740 1 -- 172.21.15.110:0/2518615784 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81a69c0 con 0x56133411c840 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.468+0000 7f76f156d740 1 -- 172.21.15.110:0/2518615784 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81a4810 con 0x56133411be50 2024-04-25T00:11:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76e7fff640 1 -- 172.21.15.110:0/2518615784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411c840 msgr2=0x561334124d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76e7fff640 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411c840 0x561334124d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76e7fff640 1 -- 172.21.15.110:0/2518615784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 msgr2=0x5613341178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76e7fff640 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x5613341178a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76e7fff640 1 -- 172.21.15.110:0/2518615784 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561334118230 con 0x56133411be50 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76e7fff640 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x5613341257c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x56133411d2b0 tx=0x7f76d8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f06dfe6448fcbf4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76cdffb640 1 -- 172.21.15.110:0/2518615784 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8020070 con 0x56133411be50 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76cdffb640 1 -- 172.21.15.110:0/2518615784 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d80206c0 con 0x56133411be50 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.469+0000 7f76cdffb640 1 -- 172.21.15.110:0/2518615784 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8010bc0 con 0x56133411be50 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.470+0000 7f76f156d740 1 -- 172.21.15.110:0/2518615784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 msgr2=0x5613341257c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.470+0000 7f76f156d740 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x5613341257c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x56133411d2b0 tx=0x7f76d8005f90 comp rx=0 tx=0).stop 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.470+0000 7f76f156d740 1 -- 172.21.15.110:0/2518615784 shutdown_connections 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.470+0000 7f76f156d740 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613341219b0 0x5613341178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.470+0000 7f76f156d740 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56133411c840 0x561334124d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.470+0000 7f76f156d740 1 --2- 172.21.15.110:0/2518615784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x5613341257c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.470+0000 7f76f156d740 1 -- 172.21.15.110:0/2518615784 >> 172.21.15.110:0/2518615784 conn(0x5613341174c0 msgr2=0x5613341122d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.471+0000 7f76f156d740 1 -- 172.21.15.110:0/2518615784 shutdown_connections 2024-04-25T00:11:48.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.471+0000 7f76f156d740 1 -- 172.21.15.110:0/2518615784 wait complete. 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.471+0000 7f76f156d740 1 -- start start 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.471+0000 7f76f156d740 1 Processor -- start 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76f156d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x56133413d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76f156d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56133411c840 0x56133413d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76f156d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613341219b0 0x56133413dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76f156d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81a4e80 con 0x56133411c840 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76f156d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81a62f0 con 0x5613341219b0 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76f156d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81a5550 con 0x56133411be50 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76f156d740 10 client.0 _ll_register_callbacks cb 0x7f76dc09aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76ecdfa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613341219b0 0x56133413dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x56133413d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x56133413d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:57220/0 (socket says 172.21.15.110:57220) 2024-04-25T00:11:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76ecdfa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613341219b0 0x56133413dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:59252/0 (socket says 172.21.15.110:59252) 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e7fff640 1 -- 172.21.15.110:0/4053487811 learned_addr learned my addr 172.21.15.110:0/4053487811 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e77fe640 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56133411c840 0x56133413d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e7fff640 1 -- 172.21.15.110:0/4053487811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613341219b0 msgr2=0x56133413dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e7fff640 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613341219b0 0x56133413dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e7fff640 1 -- 172.21.15.110:0/4053487811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56133411c840 msgr2=0x56133413d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e7fff640 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56133411c840 0x56133413d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.472+0000 7f76e7fff640 1 -- 172.21.15.110:0/4053487811 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561334196d10 con 0x56133411be50 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.473+0000 7f76c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.473+0000 7f76e7fff640 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x56133413d200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x56133411d2b0 tx=0x7f76d80056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43b5f6902fc641ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.473+0000 7f76c77fe640 1 -- 172.21.15.110:0/4053487811 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8003f50 con 0x56133411be50 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.473+0000 7f76c77fe640 1 -- 172.21.15.110:0/4053487811 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f76d8020040 con 0x56133411be50 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.473+0000 7f76f156d740 1 -- 172.21.15.110:0/4053487811 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d80206c0 con 0x56133411be50 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.473+0000 7f76c77fe640 1 -- 172.21.15.110:0/4053487811 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f76d8020240 con 0x56133411be50 2024-04-25T00:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.473+0000 7f76f156d740 10 client.25591 fetch_fsmap learned FSMap version 32 2024-04-25T00:11:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.473+0000 7f76f156d740 1 -- 172.21.15.110:0/4053487811 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561334198000 con 0x56133411be50 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76c77fe640 1 -- 172.21.15.110:0/4053487811 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d80206c0 con 0x56133411be50 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76c77fe640 1 -- 172.21.15.110:0/4053487811 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f76d8029610 con 0x56133411be50 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76f156d740 10 client.25591 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76f156d740 10 client.25591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76f156d740 20 client.25591 populate_metadata read hostname 'smithi110' 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76f156d740 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x561334132080 0x56133419a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76f156d740 4 client.25591 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76f156d740 1 -- 172.21.15.110:0/4053487811 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561334087540 con 0x561334132080 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.474+0000 7f76e77fe640 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x561334132080 0x56133419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.475+0000 7f76e77fe640 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x561334132080 0x56133419a970 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6c51f6f1426956 server_cookie=c8cfac06d1126bd9 in_seq=0 out_seq=0 2024-04-25T00:11:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.475+0000 7f76c77fe640 10 client.25591 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76c77fe640 1 -- 172.21.15.110:0/4053487811 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561334087540 con 0x561334132080 2024-04-25T00:11:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76c77fe640 10 client.25591 handle_command_reply: tid=0 2024-04-25T00:11:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 client.25591 shutdown 2024-04-25T00:11:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 -- 172.21.15.110:0/4053487811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 msgr2=0x56133413d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x56133413d200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x56133411d2b0 tx=0x7f76d80056d0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 -- 172.21.15.110:0/4053487811 shutdown_connections 2024-04-25T00:11:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x561334132080 0x56133419a970 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613341219b0 0x56133413dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56133411c840 0x56133413d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 --2- 172.21.15.110:0/4053487811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56133411be50 0x56133413d200 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 -- 172.21.15.110:0/4053487811 >> 172.21.15.110:0/4053487811 conn(0x5613341174c0 msgr2=0x5613341178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76c77fe640 0 client.25591 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 -- 172.21.15.110:0/4053487811 shutdown_connections 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 1 -- 172.21.15.110:0/4053487811 wait complete. 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.491+0000 7f76f156d740 20 client.25591 trim_cache size 0 max 16384 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:11:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:11:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.492+0000 7f76ee85d640 1 -- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76d00a2d30 msgr2=0x7f76d00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.492+0000 7f76ee85d640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76d00a2d30 0x7f76d00a51f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f76e0006010 tx=0x7f76e0048000 comp rx=0 tx=0).stop 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.492+0000 7f76ee85d640 1 -- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8075780 msgr2=0x7f76e819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.492+0000 7f76ee85d640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8075780 0x7f76e819a040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00b4a0 tx=0x7f76dc00b960 comp rx=0 tx=0).stop 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.493+0000 7f76ee85d640 1 -- 172.21.15.110:0/4151469332 shutdown_connections 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.493+0000 7f76ee85d640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f76d00a2d30 0x7f76d00a51f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.493+0000 7f76ee85d640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e8109660 0x7f76e81a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.493+0000 7f76ee85d640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76e8075780 0x7f76e819a040 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.493+0000 7f76ee85d640 1 --2- 172.21.15.110:0/4151469332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e80741a0 0x7f76e8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.493+0000 7f76ee85d640 1 -- 172.21.15.110:0/4151469332 >> 172.21.15.110:0/4151469332 conn(0x7f76e80fcd30 msgr2=0x7f76e8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.493+0000 7f76ee85d640 1 -- 172.21.15.110:0/4151469332 shutdown_connections 2024-04-25T00:11:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:11:48.493+0000 7f76ee85d640 1 -- 172.21.15.110:0/4151469332 wait complete. 2024-04-25T00:11:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:49 smithi188 ceph-mon[33285]: pgmap v644: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 968 op/s 2024-04-25T00:11:50.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:49 smithi110 ceph-mon[29187]: pgmap v644: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 968 op/s 2024-04-25T00:11:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:49 smithi173 ceph-mon[32402]: pgmap v644: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 968 op/s 2024-04-25T00:11:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 525.0 s, 818.8 tps, lat 121.977 ms stddev 18.768 2024-04-25T00:11:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:51 smithi188 ceph-mon[33285]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 MiB/s wr, 799 op/s 2024-04-25T00:11:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:51 smithi110 ceph-mon[29187]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 MiB/s wr, 799 op/s 2024-04-25T00:11:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:51 smithi173 ceph-mon[32402]: pgmap v645: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 MiB/s wr, 799 op/s 2024-04-25T00:11:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:53 smithi188 ceph-mon[33285]: pgmap v646: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 951 op/s 2024-04-25T00:11:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:53 smithi110 ceph-mon[29187]: pgmap v646: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 951 op/s 2024-04-25T00:11:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:53 smithi173 ceph-mon[32402]: pgmap v646: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 951 op/s 2024-04-25T00:11:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:55 smithi188 ceph-mon[33285]: pgmap v647: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 814 op/s 2024-04-25T00:11:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:55 smithi110 ceph-mon[29187]: pgmap v647: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 814 op/s 2024-04-25T00:11:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:55 smithi173 ceph-mon[32402]: pgmap v647: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 814 op/s 2024-04-25T00:11:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 530.0 s, 815.2 tps, lat 123.102 ms stddev 22.000 2024-04-25T00:11:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:57 smithi188 ceph-mon[33285]: pgmap v648: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-04-25T00:11:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:57 smithi110 ceph-mon[29187]: pgmap v648: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-04-25T00:11:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:57 smithi173 ceph-mon[32402]: pgmap v648: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-04-25T00:12:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:11:59 smithi188 ceph-mon[33285]: pgmap v649: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 960 op/s 2024-04-25T00:12:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:11:59 smithi110 ceph-mon[29187]: pgmap v649: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 960 op/s 2024-04-25T00:12:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:11:59 smithi173 ceph-mon[32402]: pgmap v649: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 960 op/s 2024-04-25T00:12:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 535.0 s, 851.0 tps, lat 117.303 ms stddev 20.200 2024-04-25T00:12:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 -- 172.21.15.110:0/3315149442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 msgr2=0x7f3598102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 --2- 172.21.15.110:0/3315149442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 0x7f3598102c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3588009900 tx=0x7f358802a410 comp rx=0 tx=0).stop 2024-04-25T00:12:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 -- 172.21.15.110:0/3315149442 shutdown_connections 2024-04-25T00:12:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 --2- 172.21.15.110:0/3315149442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35981031b0 0x7f3598109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 --2- 172.21.15.110:0/3315149442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 0x7f3598102c70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 --2- 172.21.15.110:0/3315149442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598101600 0x7f3598101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 -- 172.21.15.110:0/3315149442 >> 172.21.15.110:0/3315149442 conn(0x7f35980fcb70 msgr2=0x7f35980fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 -- 172.21.15.110:0/3315149442 shutdown_connections 2024-04-25T00:12:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.627+0000 7f3597fff640 1 -- 172.21.15.110:0/3315149442 wait complete. 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.628+0000 7f3597fff640 1 Processor -- start 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f3597fff640 1 -- start start 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f3597fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598101600 0x7f35981976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f3597fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 0x7f3598197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f3597fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35981031b0 0x7f359819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f3597fff640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f359810d1b0 con 0x7f35981031b0 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f3597fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f359810d030 con 0x7f35981027f0 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f3597fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f359810d330 con 0x7f3598101600 2024-04-25T00:12:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f35967fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 0x7f3598197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f35967fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 0x7f3598197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:34530/0 (socket says 172.21.15.110:34530) 2024-04-25T00:12:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f35977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35981031b0 0x7f359819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f35967fc640 1 -- 172.21.15.110:0/386183606 learned_addr learned my addr 172.21.15.110:0/386183606 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.629+0000 7f3596ffd640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598101600 0x7f35981976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f3596ffd640 1 -- 172.21.15.110:0/386183606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 msgr2=0x7f3598197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f3596ffd640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 0x7f3598197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f3596ffd640 1 -- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35981031b0 msgr2=0x7f359819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f3596ffd640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35981031b0 0x7f359819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f3596ffd640 1 -- 172.21.15.110:0/386183606 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35980739f0 con 0x7f3598101600 2024-04-25T00:12:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f35977fe640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35981031b0 0x7f359819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f35967fc640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 0x7f3598197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f3596ffd640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598101600 0x7f35981976a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3590005fd0 tx=0x7f359000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f357ffff640 1 -- 172.21.15.110:0/386183606 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590003970 con 0x7f3598101600 2024-04-25T00:12:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f357ffff640 1 -- 172.21.15.110:0/386183606 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3590004020 con 0x7f3598101600 2024-04-25T00:12:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.630+0000 7f3597fff640 1 -- 172.21.15.110:0/386183606 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3598073ce0 con 0x7f3598101600 2024-04-25T00:12:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.631+0000 7f357ffff640 1 -- 172.21.15.110:0/386183606 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590010720 con 0x7f3598101600 2024-04-25T00:12:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.631+0000 7f3597fff640 1 -- 172.21.15.110:0/386183606 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35980741d0 con 0x7f3598101600 2024-04-25T00:12:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.634+0000 7f357ffff640 1 -- 172.21.15.110:0/386183606 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f35900041d0 con 0x7f3598101600 2024-04-25T00:12:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.635+0000 7f35a0009740 1 Processor -- start 2024-04-25T00:12:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.635+0000 7f357ffff640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2c80 0x7f35800a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.635+0000 7f357ffff640 1 -- 172.21.15.110:0/386183606 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f35900cf850 con 0x7f3598101600 2024-04-25T00:12:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35a0009740 1 -- start start 2024-04-25T00:12:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35977fe640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2c80 0x7f35800a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35a0009740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363e3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35a0009740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3a20 0x55db363e8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35a0009740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35a0009740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55db36346cb0 con 0x55db363e3a20 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35a0009740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55db362ef5b0 con 0x55db363e3030 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35a0009740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55db36342c70 con 0x55db363e8b90 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f3596ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f3596ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363e3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:34544/0 (socket says 172.21.15.110:34544) 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f3596ffd640 1 -- 172.21.15.110:0/3573783412 learned_addr learned my addr 172.21.15.110:0/3573783412 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.636+0000 7f35967fc640 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3a20 0x55db363e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f3596ffd640 1 -- 172.21.15.110:0/3573783412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 msgr2=0x55db363eb020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:12:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f3596ffd640 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363eb020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f3596ffd640 1 -- 172.21.15.110:0/3573783412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3a20 msgr2=0x55db363e8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f3596ffd640 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3a20 0x55db363e8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f35977fe640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2c80 0x7f35800a5140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f358c009ba0 tx=0x7f358c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f3596ffd640 1 -- 172.21.15.110:0/3573783412 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db3605b210 con 0x55db363e3030 2024-04-25T00:12:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f35967fc640 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3a20 0x55db363e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f3596ffd640 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363e3450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f35900a47c0 tx=0x7f3590004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb683aa08223e151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.637+0000 7f357d7fa640 1 -- 172.21.15.110:0/3573783412 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900a0430 con 0x55db363e3030 2024-04-25T00:12:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.638+0000 7f357d7fa640 1 -- 172.21.15.110:0/3573783412 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35900b5070 con 0x55db363e3030 2024-04-25T00:12:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.638+0000 7f357d7fa640 1 -- 172.21.15.110:0/3573783412 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900986f0 con 0x55db363e3030 2024-04-25T00:12:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 -- 172.21.15.110:0/3573783412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 msgr2=0x55db363e3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363e3450 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f35900a47c0 tx=0x7f3590004560 comp rx=0 tx=0).stop 2024-04-25T00:12:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 -- 172.21.15.110:0/3573783412 shutdown_connections 2024-04-25T00:12:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3a20 0x55db363e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 --2- 172.21.15.110:0/3573783412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363e3450 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 -- 172.21.15.110:0/3573783412 >> 172.21.15.110:0/3573783412 conn(0x55db363de6a0 msgr2=0x55db363deaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 -- 172.21.15.110:0/3573783412 shutdown_connections 2024-04-25T00:12:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.639+0000 7f35a0009740 1 -- 172.21.15.110:0/3573783412 wait complete. 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.640+0000 7f35a0009740 1 -- start start 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.640+0000 7f35a0009740 1 Processor -- start 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35a0009740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35a0009740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3a20 0x55db363f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35a0009740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f3596ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35a0009740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55db363f4350 con 0x55db363e8b90 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f3596ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:34558/0 (socket says 172.21.15.110:34558) 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f3596ffd640 1 -- 172.21.15.110:0/312341498 learned_addr learned my addr 172.21.15.110:0/312341498 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55db36302b50 con 0x55db363e3030 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55db363f44d0 con 0x55db363e3a20 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35967fc640 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3a20 0x55db363f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35977fe640 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f35a0009740 10 client.0 _ll_register_callbacks cb 0x55db3633dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f3596ffd640 1 -- 172.21.15.110:0/312341498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3a20 msgr2=0x55db363f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f3596ffd640 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3a20 0x55db363f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f3596ffd640 1 -- 172.21.15.110:0/312341498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 msgr2=0x55db363fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f3596ffd640 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.641+0000 7f3596ffd640 1 -- 172.21.15.110:0/312341498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db36127620 con 0x55db363e3030 2024-04-25T00:12:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.642+0000 7f3596ffd640 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363f5d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f35900049d0 tx=0x7f3590005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7513fd29127f393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.642+0000 7f357aff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.642+0000 7f357aff5640 1 -- 172.21.15.110:0/312341498 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900a4860 con 0x55db363e3030 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.642+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35900b5070 con 0x55db363e3030 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.642+0000 7f357aff5640 1 -- 172.21.15.110:0/312341498 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f35900a4a00 con 0x55db363e3030 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.642+0000 7f357aff5640 1 -- 172.21.15.110:0/312341498 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3590003b10 con 0x55db363e3030 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.642+0000 7f35a0009740 10 client.25619 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.642+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db36400650 con 0x55db363e3030 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f357aff5640 1 -- 172.21.15.110:0/312341498 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35900b5070 con 0x55db363e3030 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f357aff5640 1 -- 172.21.15.110:0/312341498 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f359009af40 con 0x55db363e3030 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f35a0009740 10 client.25619 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f35a0009740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f35a0009740 20 client.25619 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f35a0009740 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55db36401dd0 0x55db364041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f35a0009740 4 client.25619 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:12:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db363541f0 con 0x55db36401dd0 2024-04-25T00:12:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.643+0000 7f35967fc640 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55db36401dd0 0x55db364041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.644+0000 7f35967fc640 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55db36401dd0 0x55db364041e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10ba5d1633817d2b server_cookie=14a169220f7520a4 in_seq=0 out_seq=0 2024-04-25T00:12:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.644+0000 7f357aff5640 10 client.25619 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.645+0000 7f357aff5640 1 -- 172.21.15.110:0/312341498 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db363541f0 con 0x55db36401dd0 2024-04-25T00:12:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.645+0000 7f357aff5640 10 client.25619 handle_command_reply: tid=0 2024-04-25T00:12:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.645+0000 7f35a0009740 1 client.25619 shutdown 2024-04-25T00:12:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.645+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 msgr2=0x55db363f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.645+0000 7f35a0009740 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363f5d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f35900049d0 tx=0x7f3590005a50 comp rx=0 tx=0).stop 2024-04-25T00:12:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 shutdown_connections 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55db36401dd0 0x55db364041e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3a20 0x55db363f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 1 --2- 172.21.15.110:0/312341498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3030 0x55db363f5d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 >> 172.21.15.110:0/312341498 conn(0x55db363de6a0 msgr2=0x55db363dd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f357aff5640 0 client.25619 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 shutdown_connections 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 1 -- 172.21.15.110:0/312341498 wait complete. 2024-04-25T00:12:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.646+0000 7f35a0009740 20 client.25619 trim_cache size 0 max 16384 2024-04-25T00:12:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.668+0000 7f35a0009740 1 Processor -- start 2024-04-25T00:12:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.668+0000 7f35a0009740 1 -- start start 2024-04-25T00:12:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.668+0000 7f35a0009740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3030 0x55db363e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.668+0000 7f35a0009740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.668+0000 7f35a0009740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.668+0000 7f35a0009740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35981a8de0 con 0x55db363e3030 2024-04-25T00:12:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.668+0000 7f35a0009740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35981a66b0 con 0x55db363e3a20 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.668+0000 7f35a0009740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35981a4500 con 0x55db363e8b90 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35967fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35977fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f3596ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3030 0x55db363e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35967fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363e5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:34566/0 (socket says 172.21.15.110:34566) 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35977fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363fd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48652/0 (socket says 172.21.15.110:48652) 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35967fc640 1 -- 172.21.15.110:0/1546099499 learned_addr learned my addr 172.21.15.110:0/1546099499 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35977fe640 1 -- 172.21.15.110:0/1546099499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 msgr2=0x55db363e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35977fe640 1 --2- 172.21.15.110:0/1546099499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35977fe640 1 -- 172.21.15.110:0/1546099499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3030 msgr2=0x55db363e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35977fe640 1 --2- 172.21.15.110:0/1546099499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3030 0x55db363e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35977fe640 1 -- 172.21.15.110:0/1546099499 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db363e8210 con 0x55db363e8b90 2024-04-25T00:12:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f35977fe640 1 --2- 172.21.15.110:0/1546099499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363fd2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f358c009640 tx=0x7f358c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4815333baf81d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.669+0000 7f357cff9640 1 -- 172.21.15.110:0/1546099499 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c029670 con 0x55db363e8b90 2024-04-25T00:12:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f357cff9640 1 -- 172.21.15.110:0/1546099499 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c024070 con 0x55db363e8b90 2024-04-25T00:12:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f357cff9640 1 -- 172.21.15.110:0/1546099499 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c01f690 con 0x55db363e8b90 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f35a0009740 1 -- 172.21.15.110:0/1546099499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 msgr2=0x55db363fd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f35a0009740 1 --2- 172.21.15.110:0/1546099499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363fd2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f358c009640 tx=0x7f358c014000 comp rx=0 tx=0).stop 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f35a0009740 1 -- 172.21.15.110:0/1546099499 shutdown_connections 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f35a0009740 1 --2- 172.21.15.110:0/1546099499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e8b90 0x55db363fd2c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f35a0009740 1 --2- 172.21.15.110:0/1546099499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f35a0009740 1 --2- 172.21.15.110:0/1546099499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e3030 0x55db363e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f35a0009740 1 -- 172.21.15.110:0/1546099499 >> 172.21.15.110:0/1546099499 conn(0x55db363de6a0 msgr2=0x55db363f4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.670+0000 7f35a0009740 1 -- 172.21.15.110:0/1546099499 shutdown_connections 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.671+0000 7f35a0009740 1 -- 172.21.15.110:0/1546099499 wait complete. 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.671+0000 7f35a0009740 1 -- start start 2024-04-25T00:12:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.671+0000 7f35a0009740 1 Processor -- start 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.671+0000 7f35a0009740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3030 0x55db363fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.671+0000 7f35a0009740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.671+0000 7f35a0009740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f3596ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3030 0x55db363fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f3596ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3030 0x55db363fcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48658/0 (socket says 172.21.15.110:48658) 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41930/0 (socket says 172.21.15.110:41930) 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35a0009740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35981a4b70 con 0x55db363e8b90 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35a0009740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35981a5fe0 con 0x55db363e3a20 2024-04-25T00:12:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35a0009740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35981a5240 con 0x55db363e3030 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35a0009740 10 client.0 _ll_register_callbacks cb 0x7f35900b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f3596ffd640 1 -- 172.21.15.110:0/1762350681 learned_addr learned my addr 172.21.15.110:0/1762350681 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35967fc640 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35977fe640 1 -- 172.21.15.110:0/1762350681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3030 msgr2=0x55db363fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35977fe640 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3030 0x55db363fcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35977fe640 1 -- 172.21.15.110:0/1762350681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 msgr2=0x55db363fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35977fe640 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35977fe640 1 -- 172.21.15.110:0/1762350681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db363f9c70 con 0x55db363e8b90 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f35977fe640 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363f9600 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f358c028000 tx=0x7f358c0296a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b603ec449f0c2131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f357aff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.672+0000 7f357aff5640 1 -- 172.21.15.110:0/1762350681 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c037060 con 0x55db363e8b90 2024-04-25T00:12:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f357aff5640 1 -- 172.21.15.110:0/1762350681 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f358c0298c0 con 0x55db363e8b90 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 1 -- 172.21.15.110:0/1762350681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358c024070 con 0x55db363e8b90 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f357aff5640 1 -- 172.21.15.110:0/1762350681 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f358c029ac0 con 0x55db363e8b90 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 10 client.16044 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 1 -- 172.21.15.110:0/1762350681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db36466f50 con 0x55db363e8b90 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f357aff5640 1 -- 172.21.15.110:0/1762350681 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c029c60 con 0x55db363e8b90 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f357aff5640 1 -- 172.21.15.110:0/1762350681 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f358c032c40 con 0x55db363e8b90 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 10 client.16044 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 20 client.16044 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55db36467860 0x55db36469c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 4 client.16044 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.673+0000 7f35a0009740 1 -- 172.21.15.110:0/1762350681 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f358c024070 con 0x55db36467860 2024-04-25T00:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.674+0000 7f3596ffd640 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55db36467860 0x55db36469c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.674+0000 7f357aff5640 10 client.16044 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.674+0000 7f3596ffd640 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55db36467860 0x55db36469c80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e761ba466495ab5b server_cookie=16b575b7ebb18822 in_seq=0 out_seq=0 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.689+0000 7f357aff5640 1 -- 172.21.15.110:0/1762350681 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f358c024070 con 0x55db36467860 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.689+0000 7f357aff5640 10 client.16044 handle_command_reply: tid=0 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.689+0000 7f35a0009740 1 client.16044 shutdown 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 -- 172.21.15.110:0/1762350681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 msgr2=0x55db363f9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363f9600 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f358c028000 tx=0x7f358c0296a0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 -- 172.21.15.110:0/1762350681 shutdown_connections 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55db36467860 0x55db36469c80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db363e8b90 0x55db363f9600 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f357aff5640 0 client.16044 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db363e3a20 0x55db363fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 --2- 172.21.15.110:0/1762350681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55db363e3030 0x55db363fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 -- 172.21.15.110:0/1762350681 >> 172.21.15.110:0/1762350681 conn(0x55db363de6a0 msgr2=0x55db363f4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 -- 172.21.15.110:0/1762350681 shutdown_connections 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 1 -- 172.21.15.110:0/1762350681 wait complete. 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.690+0000 7f35a0009740 20 client.16044 trim_cache size 0 max 16384 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:12:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:12:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 -- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2c80 msgr2=0x7f35800a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2c80 0x7f35800a5140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f358c009ba0 tx=0x7f358c005f90 comp rx=0 tx=0).stop 2024-04-25T00:12:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 -- 172.21.15.110:0/386183606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598101600 msgr2=0x7f35981976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598101600 0x7f35981976a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3590005fd0 tx=0x7f359000ef90 comp rx=0 tx=0).stop 2024-04-25T00:12:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 -- 172.21.15.110:0/386183606 shutdown_connections 2024-04-25T00:12:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f35800a2c80 0x7f35800a5140 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35981031b0 0x7f359819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35981027f0 0x7f3598197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 --2- 172.21.15.110:0/386183606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3598101600 0x7f35981976a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 -- 172.21.15.110:0/386183606 >> 172.21.15.110:0/386183606 conn(0x7f35980fcb70 msgr2=0x7f3598105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 -- 172.21.15.110:0/386183606 shutdown_connections 2024-04-25T00:12:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:01.691+0000 7f3597fff640 1 -- 172.21.15.110:0/386183606 wait complete. 2024-04-25T00:12:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:01 smithi188 ceph-mon[33285]: pgmap v650: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 806 op/s 2024-04-25T00:12:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:01 smithi110 ceph-mon[29187]: pgmap v650: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 806 op/s 2024-04-25T00:12:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:01 smithi173 ceph-mon[32402]: pgmap v650: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 806 op/s 2024-04-25T00:12:04.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:03 smithi110 ceph-mon[29187]: pgmap v651: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.1 MiB/s wr, 958 op/s 2024-04-25T00:12:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:03 smithi173 ceph-mon[32402]: pgmap v651: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.1 MiB/s wr, 958 op/s 2024-04-25T00:12:04.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:03 smithi188 ceph-mon[33285]: pgmap v651: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.1 MiB/s wr, 958 op/s 2024-04-25T00:12:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:05 smithi110 ceph-mon[29187]: pgmap v652: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 870 op/s 2024-04-25T00:12:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:05 smithi173 ceph-mon[32402]: pgmap v652: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 870 op/s 2024-04-25T00:12:06.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:05 smithi188 ceph-mon[33285]: pgmap v652: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 870 op/s 2024-04-25T00:12:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 540.0 s, 835.6 tps, lat 119.838 ms stddev 14.166 2024-04-25T00:12:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:07 smithi110 ceph-mon[29187]: pgmap v653: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 789 op/s 2024-04-25T00:12:08.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:07 smithi188 ceph-mon[33285]: pgmap v653: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 789 op/s 2024-04-25T00:12:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:07 smithi173 ceph-mon[32402]: pgmap v653: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 789 op/s 2024-04-25T00:12:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:09 smithi110 ceph-mon[29187]: pgmap v654: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 950 op/s 2024-04-25T00:12:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:09 smithi173 ceph-mon[32402]: pgmap v654: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 950 op/s 2024-04-25T00:12:10.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:09 smithi188 ceph-mon[33285]: pgmap v654: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.1 MiB/s wr, 950 op/s 2024-04-25T00:12:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 545.0 s, 853.6 tps, lat 116.961 ms stddev 14.984 2024-04-25T00:12:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:11 smithi110 ceph-mon[29187]: pgmap v655: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 820 op/s 2024-04-25T00:12:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:11 smithi173 ceph-mon[32402]: pgmap v655: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 820 op/s 2024-04-25T00:12:12.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:11 smithi188 ceph-mon[33285]: pgmap v655: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 820 op/s 2024-04-25T00:12:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:13 smithi110 ceph-mon[29187]: pgmap v656: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.9 MiB/s wr, 1.05k op/s 2024-04-25T00:12:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:13 smithi173 ceph-mon[32402]: pgmap v656: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.9 MiB/s wr, 1.05k op/s 2024-04-25T00:12:14.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:13 smithi188 ceph-mon[33285]: pgmap v656: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.9 MiB/s wr, 1.05k op/s 2024-04-25T00:12:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.820+0000 7efdb159a640 1 -- 172.21.15.110:0/1954722457 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac036070 con 0x7efdb41022c0 2024-04-25T00:12:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.821+0000 7efdb8e10640 1 -- 172.21.15.110:0/1954722457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 msgr2=0x7efdb41046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.821+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1954722457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 0x7efdb41046b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efdac009be0 tx=0x7efdac02a6c0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.821+0000 7efdb8e10640 1 -- 172.21.15.110:0/1954722457 shutdown_connections 2024-04-25T00:12:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.821+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1954722457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdb4107520 0x7efdb4109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.821+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1954722457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb4104bf0 0x7efdb4106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.821+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1954722457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 0x7efdb41046b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.821+0000 7efdb8e10640 1 -- 172.21.15.110:0/1954722457 >> 172.21.15.110:0/1954722457 conn(0x7efdb40fbc50 msgr2=0x7efdb40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.822+0000 7efdb8e10640 1 -- 172.21.15.110:0/1954722457 shutdown_connections 2024-04-25T00:12:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.822+0000 7efdb8e10640 1 -- 172.21.15.110:0/1954722457 wait complete. 2024-04-25T00:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb8e10640 1 Processor -- start 2024-04-25T00:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb8e10640 1 -- start start 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb8e10640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 0x7efdb4195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb8e10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdb4104bf0 0x7efdb4195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb8e10640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb4107520 0x7efdb41a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb8e10640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efdb410d350 con 0x7efdb4104bf0 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb8e10640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efdb410d1d0 con 0x7efdb41022c0 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb8e10640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efdb410d4d0 con 0x7efdb4107520 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 0x7efdb4195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdb4104bf0 0x7efdb4195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 0x7efdb4195720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:35072/0 (socket says 172.21.15.110:35072) 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.823+0000 7efdb259c640 1 -- 172.21.15.110:0/1265452321 learned_addr learned my addr 172.21.15.110:0/1265452321 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.824+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb4107520 0x7efdb41a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.824+0000 7efdb259c640 1 -- 172.21.15.110:0/1265452321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb4107520 msgr2=0x7efdb41a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.824+0000 7efdb259c640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb4107520 0x7efdb41a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.824+0000 7efdb259c640 1 -- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdb4104bf0 msgr2=0x7efdb4195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.824+0000 7efdb259c640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdb4104bf0 0x7efdb4195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.824+0000 7efdb259c640 1 -- 172.21.15.110:0/1265452321 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb41a5b70 con 0x7efdb41022c0 2024-04-25T00:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.824+0000 7efdb1d9b640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdb4104bf0 0x7efdb4195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.825+0000 7efdb259c640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 0x7efdb4195720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdac02acb0 tx=0x7efdac030fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.825+0000 7efda37fe640 1 -- 172.21.15.110:0/1265452321 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac041070 con 0x7efdb41022c0 2024-04-25T00:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.825+0000 7efda37fe640 1 -- 172.21.15.110:0/1265452321 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdac003e20 con 0x7efdb41022c0 2024-04-25T00:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.825+0000 7efda37fe640 1 -- 172.21.15.110:0/1265452321 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac036070 con 0x7efdb41022c0 2024-04-25T00:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.825+0000 7efdb8e10640 1 -- 172.21.15.110:0/1265452321 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb41a5e00 con 0x7efdb41022c0 2024-04-25T00:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.825+0000 7efdb8e10640 1 -- 172.21.15.110:0/1265452321 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb41a61b0 con 0x7efdb41022c0 2024-04-25T00:12:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.829+0000 7efdbbbf5740 1 Processor -- start 2024-04-25T00:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.829+0000 7efdbbbf5740 1 -- start start 2024-04-25T00:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.829+0000 7efda37fe640 1 -- 172.21.15.110:0/1265452321 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efdac044070 con 0x7efdb41022c0 2024-04-25T00:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0fd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d102340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 0x55d93d104e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d93d060cb0 con 0x55d93d0fd840 2024-04-25T00:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d93d0095b0 con 0x55d93d1029b0 2024-04-25T00:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d93d05cc70 con 0x55d93d0fce50 2024-04-25T00:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb2d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb2d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:57496/0 (socket says 172.21.15.110:57496) 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb2d9d640 1 -- 172.21.15.110:0/1031202789 learned_addr learned my addr 172.21.15.110:0/1031202789 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb259c640 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb1d9b640 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 0x55d93d104e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efda37fe640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd9c0a2cb0 0x7efd9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb2d9d640 1 -- 172.21.15.110:0/1031202789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 msgr2=0x55d93d104e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 0x55d93d104e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb2d9d640 1 -- 172.21.15.110:0/1031202789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 msgr2=0x55d93d102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.830+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.831+0000 7efdb2d9d640 1 -- 172.21.15.110:0/1031202789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93cd75210 con 0x55d93d0fce50 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.831+0000 7efdb259c640 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.831+0000 7efda37fe640 1 -- 172.21.15.110:0/1265452321 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efdac0efa00 con 0x7efdb41022c0 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.831+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd9c0a2cb0 0x7efd9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.831+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0fd270 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efda400b700 tx=0x7efda400bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0d27f3c142ec2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.831+0000 7efda0ff9640 1 -- 172.21.15.110:0/1031202789 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4011440 con 0x55d93d0fce50 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.831+0000 7efda0ff9640 1 -- 172.21.15.110:0/1031202789 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efda4015070 con 0x55d93d0fce50 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.831+0000 7efda0ff9640 1 -- 172.21.15.110:0/1031202789 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda401bc00 con 0x55d93d0fce50 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.832+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd9c0a2cb0 0x7efd9c0a5170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efda401c000 tx=0x7efda4010810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.832+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1031202789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 msgr2=0x55d93d0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.832+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0fd270 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efda400b700 tx=0x7efda400bbc0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.832+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1031202789 shutdown_connections 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.832+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 0x55d93d104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.832+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.832+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1031202789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0fd270 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.832+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1031202789 >> 172.21.15.110:0/1031202789 conn(0x55d93d0f84c0 msgr2=0x55d93d0f88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.833+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1031202789 shutdown_connections 2024-04-25T00:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.833+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1031202789 wait complete. 2024-04-25T00:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 1 -- start start 2024-04-25T00:12:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 1 Processor -- start 2024-04-25T00:12:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d1072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d1077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d10e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d93d1059a0 con 0x55d93d1029b0 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d93d01cb50 con 0x55d93d0fd840 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d93d105b20 con 0x55d93d0fce50 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdbbbf5740 10 client.0 _ll_register_callbacks cb 0x55d93d057ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdb1d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d1077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdb1d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d1077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:35086/0 (socket says 172.21.15.110:35086) 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdb1d9b640 1 -- 172.21.15.110:0/1783085678 learned_addr learned my addr 172.21.15.110:0/1783085678 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d10e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.834+0000 7efdb259c640 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d1072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efdb1d9b640 1 -- 172.21.15.110:0/1783085678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 msgr2=0x55d93d1072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efdb1d9b640 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d1072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efdb1d9b640 1 -- 172.21.15.110:0/1783085678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 msgr2=0x55d93d10e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efdb1d9b640 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d10e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efdb1d9b640 1 -- 172.21.15.110:0/1783085678 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d93ce41620 con 0x55d93d0fd840 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efdb1d9b640 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d1077f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efda8012d00 tx=0x7efda800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d624970ca241fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efd99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efd99ffb640 1 -- 172.21.15.110:0/1783085678 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8012110 con 0x55d93d0fd840 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efd99ffb640 1 -- 172.21.15.110:0/1783085678 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efda8004050 con 0x55d93d0fd840 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.835+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1783085678 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda4015070 con 0x55d93d0fd840 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efd99ffb640 1 -- 172.21.15.110:0/1783085678 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efda80122b0 con 0x55d93d0fd840 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efdbbbf5740 10 client.25637 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1783085678 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d93d111ba0 con 0x55d93d0fd840 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efd99ffb640 1 -- 172.21.15.110:0/1783085678 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efda4015070 con 0x55d93d0fd840 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efd99ffb640 1 -- 172.21.15.110:0/1783085678 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efda8010430 con 0x55d93d0fd840 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efdbbbf5740 10 client.25637 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efdbbbf5740 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efdbbbf5740 20 client.25637 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d93d1132e0 0x55d93d115760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efdbbbf5740 4 client.25637 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.836+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1783085678 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d93d06e1f0 con 0x55d93d1132e0 2024-04-25T00:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.837+0000 7efdb259c640 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d93d1132e0 0x55d93d115760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.837+0000 7efd99ffb640 10 client.25637 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.839+0000 7efdb259c640 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d93d1132e0 0x55d93d115760 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73aa1961fa2a3b37 server_cookie=d2f4edc0d079e5ab in_seq=0 out_seq=0 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.839+0000 7efd99ffb640 1 -- 172.21.15.110:0/1783085678 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d93d06e1f0 con 0x55d93d1132e0 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efd99ffb640 10 client.25637 handle_command_reply: tid=0 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 client.25637 shutdown 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1783085678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 msgr2=0x55d93d1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d1077f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efda8012d00 tx=0x7efda800f790 comp rx=0 tx=0).stop 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1783085678 shutdown_connections 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d93d1132e0 0x55d93d115760 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efd99ffb640 0 client.25637 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d1077f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/1783085678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1783085678 >> 172.21.15.110:0/1783085678 conn(0x55d93d0f84c0 msgr2=0x55d93d0f75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1783085678 shutdown_connections 2024-04-25T00:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 1 -- 172.21.15.110:0/1783085678 wait complete. 2024-04-25T00:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.840+0000 7efdbbbf5740 20 client.25637 trim_cache size 0 max 16384 2024-04-25T00:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.866+0000 7efdbbbf5740 1 Processor -- start 2024-04-25T00:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.866+0000 7efdbbbf5740 1 -- start start 2024-04-25T00:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0f3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d10e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d0feca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b1940 con 0x55d93d1029b0 2024-04-25T00:12:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41af210 con 0x55d93d0fd840 2024-04-25T00:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41ad060 con 0x55d93d0fce50 2024-04-25T00:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d0feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d0feca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46328/0 (socket says 172.21.15.110:46328) 2024-04-25T00:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb1d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d10e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 -- 172.21.15.110:0/598124440 learned_addr learned my addr 172.21.15.110:0/598124440 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 -- 172.21.15.110:0/598124440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 msgr2=0x55d93d0f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/598124440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 -- 172.21.15.110:0/598124440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 msgr2=0x55d93d10e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/598124440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d10e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 -- 172.21.15.110:0/598124440 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93d0fdc20 con 0x55d93d1029b0 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.867+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/598124440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d0feca0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efda40039c0 tx=0x7efda4027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7471a12c4e541c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.868+0000 7efd9bfff640 1 -- 172.21.15.110:0/598124440 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda400be10 con 0x55d93d1029b0 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.868+0000 7efd9bfff640 1 -- 172.21.15.110:0/598124440 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efda40042d0 con 0x55d93d1029b0 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.868+0000 7efd9bfff640 1 -- 172.21.15.110:0/598124440 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda40048e0 con 0x55d93d1029b0 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 -- 172.21.15.110:0/598124440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 msgr2=0x55d93d0feca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/598124440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d0feca0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efda40039c0 tx=0x7efda4027000 comp rx=0 tx=0).stop 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 -- 172.21.15.110:0/598124440 shutdown_connections 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/598124440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d1029b0 0x55d93d0feca0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/598124440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d0fd840 0x55d93d10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/598124440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d0f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 -- 172.21.15.110:0/598124440 >> 172.21.15.110:0/598124440 conn(0x55d93d0f84c0 msgr2=0x55d93d0f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 -- 172.21.15.110:0/598124440 shutdown_connections 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.869+0000 7efdbbbf5740 1 -- 172.21.15.110:0/598124440 wait complete. 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 1 -- start start 2024-04-25T00:12:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 1 Processor -- start 2024-04-25T00:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d10b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d10bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 0x55d93d1778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41ad6d0 con 0x55d93d0fd840 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41aeb40 con 0x55d93d1029b0 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdb1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d10bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41adda0 con 0x55d93d0fce50 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdb259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d10b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdb1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d10bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46336/0 (socket says 172.21.15.110:46336) 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdb1d9b640 1 -- 172.21.15.110:0/3099502875 learned_addr learned my addr 172.21.15.110:0/3099502875 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdbbbf5740 10 client.0 _ll_register_callbacks cb 0x7efda8009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.870+0000 7efdb2d9d640 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 0x55d93d1778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdb259c640 1 -- 172.21.15.110:0/3099502875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 msgr2=0x55d93d1778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdb259c640 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 0x55d93d1778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdb259c640 1 -- 172.21.15.110:0/3099502875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 msgr2=0x55d93d10bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdb259c640 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d10bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdb259c640 1 -- 172.21.15.110:0/3099502875 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d93d177e10 con 0x55d93d0fce50 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdb259c640 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d10b6f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efdac0ed580 tx=0x7efdac0f3430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83914880601ac77f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efd99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efd99ffb640 1 -- 172.21.15.110:0/3099502875 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac0bbdd0 con 0x55d93d0fce50 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efd99ffb640 1 -- 172.21.15.110:0/3099502875 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7efdac0ba590 con 0x55d93d0fce50 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdbbbf5740 1 -- 172.21.15.110:0/3099502875 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda40042d0 con 0x55d93d0fce50 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efd99ffb640 1 -- 172.21.15.110:0/3099502875 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efdac0bbbf0 con 0x55d93d0fce50 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdbbbf5740 10 client.25624 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.871+0000 7efdbbbf5740 1 -- 172.21.15.110:0/3099502875 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d93d1790b0 con 0x55d93d0fce50 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efd99ffb640 1 -- 172.21.15.110:0/3099502875 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efda40042d0 con 0x55d93d0fce50 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efd99ffb640 1 -- 172.21.15.110:0/3099502875 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efdac0b8900 con 0x55d93d0fce50 2024-04-25T00:12:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efdbbbf5740 10 client.25624 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efdbbbf5740 10 client.25624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efdbbbf5740 20 client.25624 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d93d179a30 0x55d93d17be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efdbbbf5740 4 client.25624 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:12:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efdbbbf5740 1 -- 172.21.15.110:0/3099502875 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d93d068540 con 0x55d93d179a30 2024-04-25T00:12:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.872+0000 7efdb1d9b640 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d93d179a30 0x55d93d17be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.873+0000 7efd99ffb640 10 client.25624 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.873+0000 7efdb1d9b640 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d93d179a30 0x55d93d17be50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa799eb8a39c889 server_cookie=794efe32fceb586b in_seq=0 out_seq=0 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efd99ffb640 1 -- 172.21.15.110:0/3099502875 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d93d068540 con 0x55d93d179a30 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efd99ffb640 10 client.25624 handle_command_reply: tid=0 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 client.25624 shutdown 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 -- 172.21.15.110:0/3099502875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 msgr2=0x55d93d10b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d10b6f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efdac0ed580 tx=0x7efdac0f3430 comp rx=0 tx=0).stop 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 -- 172.21.15.110:0/3099502875 shutdown_connections 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55d93d179a30 0x55d93d17be50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d93d1029b0 0x55d93d1778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efd99ffb640 0 client.25624 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d93d0fd840 0x55d93d10bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 --2- 172.21.15.110:0/3099502875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d93d0fce50 0x55d93d10b6f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 -- 172.21.15.110:0/3099502875 >> 172.21.15.110:0/3099502875 conn(0x55d93d0f84c0 msgr2=0x55d93d107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 -- 172.21.15.110:0/3099502875 shutdown_connections 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 1 -- 172.21.15.110:0/3099502875 wait complete. 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.888+0000 7efdbbbf5740 20 client.25624 trim_cache size 0 max 16384 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:12:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:12:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 -- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd9c0a2cb0 msgr2=0x7efd9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd9c0a2cb0 0x7efd9c0a5170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efda401c000 tx=0x7efda4010810 comp rx=0 tx=0).stop 2024-04-25T00:12:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 -- 172.21.15.110:0/1265452321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 msgr2=0x7efdb4195720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 0x7efdb4195720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdac02acb0 tx=0x7efdac030fa0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 -- 172.21.15.110:0/1265452321 shutdown_connections 2024-04-25T00:12:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7efd9c0a2cb0 0x7efd9c0a5170 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb4107520 0x7efdb41a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdb4104bf0 0x7efdb4195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 --2- 172.21.15.110:0/1265452321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efdb41022c0 0x7efdb4195720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 -- 172.21.15.110:0/1265452321 >> 172.21.15.110:0/1265452321 conn(0x7efdb40fbc50 msgr2=0x7efdb4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 -- 172.21.15.110:0/1265452321 shutdown_connections 2024-04-25T00:12:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:14.889+0000 7efdb8e10640 1 -- 172.21.15.110:0/1265452321 wait complete. 2024-04-25T00:12:16.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:15 smithi110 ceph-mon[29187]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 863 op/s 2024-04-25T00:12:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:15 smithi173 ceph-mon[32402]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 863 op/s 2024-04-25T00:12:16.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:15 smithi188 ceph-mon[33285]: pgmap v657: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 863 op/s 2024-04-25T00:12:16.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 550.0 s, 885.4 tps, lat 112.788 ms stddev 13.577 2024-04-25T00:12:18.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:17 smithi110 ceph-mon[29187]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 785 op/s 2024-04-25T00:12:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:17 smithi173 ceph-mon[32402]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 785 op/s 2024-04-25T00:12:18.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:17 smithi188 ceph-mon[33285]: pgmap v658: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 785 op/s 2024-04-25T00:12:20.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:19 smithi110 ceph-mon[29187]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 1.06k op/s 2024-04-25T00:12:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:19 smithi173 ceph-mon[32402]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 1.06k op/s 2024-04-25T00:12:20.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:19 smithi188 ceph-mon[33285]: pgmap v659: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 1.06k op/s 2024-04-25T00:12:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 555.0 s, 864.6 tps, lat 115.549 ms stddev 14.694 2024-04-25T00:12:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:21 smithi173 ceph-mon[32402]: pgmap v660: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 934 op/s 2024-04-25T00:12:22.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:21 smithi188 ceph-mon[33285]: pgmap v660: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 934 op/s 2024-04-25T00:12:22.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:21 smithi110 ceph-mon[29187]: pgmap v660: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 934 op/s 2024-04-25T00:12:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:23 smithi173 ceph-mon[32402]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.0 MiB/s wr, 1.06k op/s 2024-04-25T00:12:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:23 smithi188 ceph-mon[33285]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.0 MiB/s wr, 1.06k op/s 2024-04-25T00:12:24.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:23 smithi110 ceph-mon[29187]: pgmap v661: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.0 MiB/s wr, 1.06k op/s 2024-04-25T00:12:25.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:24 smithi173 ceph-mon[32402]: pgmap v662: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 928 op/s 2024-04-25T00:12:25.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:24 smithi188 ceph-mon[33285]: pgmap v662: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 928 op/s 2024-04-25T00:12:25.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:24 smithi110 ceph-mon[29187]: pgmap v662: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 928 op/s 2024-04-25T00:12:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 560.0 s, 827.2 tps, lat 121.365 ms stddev 16.324 2024-04-25T00:12:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:27 smithi188 ceph-mon[33285]: pgmap v663: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 856 op/s 2024-04-25T00:12:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.034+0000 7fce8adab640 1 -- 172.21.15.110:0/1869221362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce841029a0 msgr2=0x7fce84102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.034+0000 7fce8adab640 1 --2- 172.21.15.110:0/1869221362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce841029a0 0x7fce84102e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009900 tx=0x7fce7c02a410 comp rx=0 tx=0).stop 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 -- 172.21.15.110:0/1869221362 shutdown_connections 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 --2- 172.21.15.110:0/1869221362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce84103360 0x7fce84109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 --2- 172.21.15.110:0/1869221362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce841029a0 0x7fce84102e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 --2- 172.21.15.110:0/1869221362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce841017b0 0x7fce84101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 -- 172.21.15.110:0/1869221362 >> 172.21.15.110:0/1869221362 conn(0x7fce840fcd00 msgr2=0x7fce840ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 -- 172.21.15.110:0/1869221362 shutdown_connections 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 -- 172.21.15.110:0/1869221362 wait complete. 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 Processor -- start 2024-04-25T00:12:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.035+0000 7fce8adab640 1 -- start start 2024-04-25T00:12:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce8adab640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce841017b0 0x7fce84197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce8adab640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce841029a0 0x7fce84197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce8adab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce84103360 0x7fce8419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce8adab640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410d2c0 con 0x7fce84103360 2024-04-25T00:12:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce8adab640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410d140 con 0x7fce841017b0 2024-04-25T00:12:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce8adab640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410d440 con 0x7fce841029a0 2024-04-25T00:12:28.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:27 smithi110 ceph-mon[29187]: pgmap v663: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 856 op/s 2024-04-25T00:12:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce89348640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce84103360 0x7fce8419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce89348640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce84103360 0x7fce8419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54296/0 (socket says 172.21.15.110:54296) 2024-04-25T00:12:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce89348640 1 -- 172.21.15.110:0/3039079448 learned_addr learned my addr 172.21.15.110:0/3039079448 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce88b47640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce841017b0 0x7fce84197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce83fff640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce841029a0 0x7fce84197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce89348640 1 -- 172.21.15.110:0/3039079448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce841029a0 msgr2=0x7fce84197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce89348640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce841029a0 0x7fce84197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce89348640 1 -- 172.21.15.110:0/3039079448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce841017b0 msgr2=0x7fce84197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce89348640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce841017b0 0x7fce84197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.036+0000 7fce89348640 1 -- 172.21.15.110:0/3039079448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce84073aa0 con 0x7fce84103360 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.037+0000 7fce89348640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce84103360 0x7fce8419eec0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fce7400eb10 tx=0x7fce7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.037+0000 7fce81ffb640 1 -- 172.21.15.110:0/3039079448 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74004b20 con 0x7fce84103360 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.037+0000 7fce81ffb640 1 -- 172.21.15.110:0/3039079448 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce74004cc0 con 0x7fce84103360 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.037+0000 7fce8adab640 1 -- 172.21.15.110:0/3039079448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce84073d90 con 0x7fce84103360 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.037+0000 7fce81ffb640 1 -- 172.21.15.110:0/3039079448 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74010690 con 0x7fce84103360 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.037+0000 7fce8adab640 1 -- 172.21.15.110:0/3039079448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce840742b0 con 0x7fce84103360 2024-04-25T00:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.040+0000 7fce81ffb640 1 -- 172.21.15.110:0/3039079448 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fce74005020 con 0x7fce84103360 2024-04-25T00:12:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce8db8b740 1 Processor -- start 2024-04-25T00:12:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce8db8b740 1 -- start start 2024-04-25T00:12:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56391955fe50 0x563919560270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x563919565340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639195659b0 0x563919567e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce8db8b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5639194c3cb0 con 0x56391955fe50 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce8db8b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56391946c5b0 con 0x563919560840 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce8db8b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639194bfc70 con 0x5639195659b0 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce81ffb640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce700a2da0 0x7fce700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce83fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56391955fe50 0x563919560270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce83fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56391955fe50 0x563919560270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54308/0 (socket says 172.21.15.110:54308) 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce83fff640 1 -- 172.21.15.110:0/420415522 learned_addr learned my addr 172.21.15.110:0/420415522 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce88b47640 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x563919565340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.041+0000 7fce83fff640 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639195659b0 0x563919567e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce89348640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce700a2da0 0x7fce700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce83fff640 1 -- 172.21.15.110:0/420415522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639195659b0 msgr2=0x563919567e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce83fff640 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639195659b0 0x563919567e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce83fff640 1 -- 172.21.15.110:0/420415522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 msgr2=0x563919565340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce83fff640 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x563919565340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce83fff640 1 -- 172.21.15.110:0/420415522 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639191d8210 con 0x56391955fe50 2024-04-25T00:12:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce83fff640 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56391955fe50 0x563919560270 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005c40 tx=0x7fce7c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf0f096e8c6a63d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce7affd640 1 -- 172.21.15.110:0/420415522 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c035070 con 0x56391955fe50 2024-04-25T00:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce7affd640 1 -- 172.21.15.110:0/420415522 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7c031630 con 0x56391955fe50 2024-04-25T00:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce7affd640 1 -- 172.21.15.110:0/420415522 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c003af0 con 0x56391955fe50 2024-04-25T00:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.042+0000 7fce81ffb640 1 -- 172.21.15.110:0/3039079448 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fce740cf4a0 con 0x7fce84103360 2024-04-25T00:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.043+0000 7fce89348640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce700a2da0 0x7fce700a5260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fce74098700 tx=0x7fce740978a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.043+0000 7fce8db8b740 1 -- 172.21.15.110:0/420415522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56391955fe50 msgr2=0x563919560270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.043+0000 7fce8db8b740 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56391955fe50 0x563919560270 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005c40 tx=0x7fce7c005e50 comp rx=0 tx=0).stop 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 -- 172.21.15.110:0/420415522 shutdown_connections 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639195659b0 0x563919567e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x563919565340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 --2- 172.21.15.110:0/420415522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56391955fe50 0x563919560270 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 -- 172.21.15.110:0/420415522 >> 172.21.15.110:0/420415522 conn(0x56391955b4c0 msgr2=0x56391955b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 -- 172.21.15.110:0/420415522 shutdown_connections 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 -- 172.21.15.110:0/420415522 wait complete. 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 -- start start 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.044+0000 7fce8db8b740 1 Processor -- start 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x56391956a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x563919571850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce8db8b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563919568900 con 0x563919560840 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce8db8b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56391947fb50 con 0x5639195659b0 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce8db8b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563919568a80 con 0x56391955fe50 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce8db8b740 10 client.0 _ll_register_callbacks cb 0x5639194baca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:43260/0 (socket says 172.21.15.110:43260) 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 -- 172.21.15.110:0/1637228079 learned_addr learned my addr 172.21.15.110:0/1637228079 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce88b47640 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x56391956a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x563919571850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 -- 172.21.15.110:0/1637228079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 msgr2=0x563919571850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x563919571850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 -- 172.21.15.110:0/1637228079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 msgr2=0x56391956a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x56391956a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.045+0000 7fce83fff640 1 -- 172.21.15.110:0/1637228079 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639192a4620 con 0x56391955fe50 2024-04-25T00:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce83fff640 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a2d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fce7c031a30 tx=0x7fce7c0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d29ba4e70663484b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce63fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce63fff640 1 -- 172.21.15.110:0/1637228079 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c035040 con 0x56391955fe50 2024-04-25T00:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce63fff640 1 -- 172.21.15.110:0/1637228079 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fce7c03baa0 con 0x56391955fe50 2024-04-25T00:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce8db8b740 1 -- 172.21.15.110:0/1637228079 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce7c031630 con 0x56391955fe50 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce63fff640 1 -- 172.21.15.110:0/1637228079 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fce7c03bca0 con 0x56391955fe50 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce8db8b740 10 client.25636 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce8db8b740 1 -- 172.21.15.110:0/1637228079 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563919568900 con 0x56391955fe50 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.046+0000 7fce63fff640 1 -- 172.21.15.110:0/1637228079 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7c031630 con 0x56391955fe50 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce63fff640 1 -- 172.21.15.110:0/1637228079 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fce7c03d450 con 0x56391955fe50 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce8db8b740 10 client.25636 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce8db8b740 10 client.25636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce8db8b740 20 client.25636 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563919575db0 0x5639195781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce8db8b740 4 client.25636 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce8db8b740 1 -- 172.21.15.110:0/1637228079 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639194d11f0 con 0x563919575db0 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce88b47640 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563919575db0 0x5639195781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce88b47640 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563919575db0 0x5639195781a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d43fcbb801e617 server_cookie=aa93ec130093360 in_seq=0 out_seq=0 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.047+0000 7fce63fff640 10 client.25636 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce63fff640 1 -- 172.21.15.110:0/1637228079 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639194d11f0 con 0x563919575db0 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce63fff640 10 client.25636 handle_command_reply: tid=0 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 client.25636 shutdown 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 -- 172.21.15.110:0/1637228079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 msgr2=0x56391956a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a2d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fce7c031a30 tx=0x7fce7c0035f0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 -- 172.21.15.110:0/1637228079 shutdown_connections 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x563919575db0 0x5639195781a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce63fff640 0 client.25636 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x563919571850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x56391956a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1637228079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a2d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.048+0000 7fce8db8b740 1 -- 172.21.15.110:0/1637228079 >> 172.21.15.110:0/1637228079 conn(0x56391955b4c0 msgr2=0x56391955a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.049+0000 7fce8db8b740 1 -- 172.21.15.110:0/1637228079 shutdown_connections 2024-04-25T00:12:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.049+0000 7fce8db8b740 1 -- 172.21.15.110:0/1637228079 wait complete. 2024-04-25T00:12:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.049+0000 7fce8db8b740 20 client.25636 trim_cache size 0 max 16384 2024-04-25T00:12:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.072+0000 7fce8db8b740 1 Processor -- start 2024-04-25T00:12:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.072+0000 7fce8db8b740 1 -- start start 2024-04-25T00:12:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x5639195717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x56391955eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce8db8b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce841b1800 con 0x563919560840 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce8db8b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce841af0d0 con 0x5639195659b0 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce8db8b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce841a3500 con 0x56391955fe50 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x5639195717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x5639195717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54336/0 (socket says 172.21.15.110:54336) 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 -- 172.21.15.110:0/1729734688 learned_addr learned my addr 172.21.15.110:0/1729734688 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce83fff640 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce83fff640 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x56391955eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 -- 172.21.15.110:0/1729734688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 msgr2=0x56391956a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 -- 172.21.15.110:0/1729734688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 msgr2=0x56391955eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x56391955eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 -- 172.21.15.110:0/1729734688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391955ad40 con 0x563919560840 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce83fff640 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x56391955eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.073+0000 7fce88b47640 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x5639195717e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009e50 tx=0x7fce6c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdf9cee3bf43658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.074+0000 7fce7a7fc640 1 -- 172.21.15.110:0/1729734688 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0043d0 con 0x563919560840 2024-04-25T00:12:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.074+0000 7fce7a7fc640 1 -- 172.21.15.110:0/1729734688 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce6c0054d0 con 0x563919560840 2024-04-25T00:12:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.074+0000 7fce7a7fc640 1 -- 172.21.15.110:0/1729734688 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c01f050 con 0x563919560840 2024-04-25T00:12:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 -- 172.21.15.110:0/1729734688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 msgr2=0x5639195717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x5639195717e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009e50 tx=0x7fce6c005f90 comp rx=0 tx=0).stop 2024-04-25T00:12:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 -- 172.21.15.110:0/1729734688 shutdown_connections 2024-04-25T00:12:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639195659b0 0x56391955eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563919560840 0x5639195717e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1729734688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 -- 172.21.15.110:0/1729734688 >> 172.21.15.110:0/1729734688 conn(0x56391955b4c0 msgr2=0x56391955e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 -- 172.21.15.110:0/1729734688 shutdown_connections 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 -- 172.21.15.110:0/1729734688 wait complete. 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 -- start start 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.075+0000 7fce8db8b740 1 Processor -- start 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x56391956eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce8db8b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639195659b0 0x56391956fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce8db8b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fce841ad590 con 0x5639195659b0 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce8db8b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce841aea00 con 0x563919560840 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce8db8b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce841adc60 con 0x56391955fe50 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce8db8b740 10 client.0 _ll_register_callbacks cb 0x7fce6c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x56391956eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce83fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639195659b0 0x56391956fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x56391956eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:52286/0 (socket says 172.21.15.110:52286) 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 -- 172.21.15.110:0/1561123191 learned_addr learned my addr 172.21.15.110:0/1561123191 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce83fff640 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 -- 172.21.15.110:0/1561123191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 msgr2=0x56391956e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 -- 172.21.15.110:0/1561123191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639195659b0 msgr2=0x56391956fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639195659b0 0x56391956fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 -- 172.21.15.110:0/1561123191 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563919570480 con 0x563919560840 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce88b47640 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x56391956eef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009f40 tx=0x7fce6c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd9294554e5e781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce63fff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.076+0000 7fce63fff640 1 -- 172.21.15.110:0/1561123191 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c004490 con 0x563919560840 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce63fff640 1 -- 172.21.15.110:0/1561123191 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fce6c004700 con 0x563919560840 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 1 -- 172.21.15.110:0/1561123191 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce6c0054d0 con 0x563919560840 2024-04-25T00:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce63fff640 1 -- 172.21.15.110:0/1561123191 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fce6c004900 con 0x563919560840 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 10 client.25661 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 1 -- 172.21.15.110:0/1561123191 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639195717c0 con 0x563919560840 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce63fff640 1 -- 172.21.15.110:0/1561123191 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce6c0054d0 con 0x563919560840 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce63fff640 1 -- 172.21.15.110:0/1561123191 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fce6c01f750 con 0x563919560840 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 10 client.25661 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 20 client.25661 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5639195dc670 0x5639195dea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 4 client.25661 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce8db8b740 1 -- 172.21.15.110:0/1561123191 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639194cb540 con 0x5639195dc670 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.077+0000 7fce83fff640 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5639195dc670 0x5639195dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.078+0000 7fce63fff640 10 client.25661 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.078+0000 7fce83fff640 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5639195dc670 0x5639195dea90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcde2c20c8d07376 server_cookie=f400ef6c82cd8749 in_seq=0 out_seq=0 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.100+0000 7fce63fff640 1 -- 172.21.15.110:0/1561123191 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639194cb540 con 0x5639195dc670 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.100+0000 7fce63fff640 10 client.25661 handle_command_reply: tid=0 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.100+0000 7fce8db8b740 1 client.25661 shutdown 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.100+0000 7fce8db8b740 1 -- 172.21.15.110:0/1561123191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 msgr2=0x56391956eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.100+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x56391956eef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009f40 tx=0x7fce6c004040 comp rx=0 tx=0).stop 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 1 -- 172.21.15.110:0/1561123191 shutdown_connections 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5639195dc670 0x5639195dea90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639195659b0 0x56391956fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563919560840 0x56391956eef0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce63fff640 0 client.25661 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 1 --2- 172.21.15.110:0/1561123191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56391955fe50 0x56391956e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 1 -- 172.21.15.110:0/1561123191 >> 172.21.15.110:0/1561123191 conn(0x56391955b4c0 msgr2=0x56391955a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 1 -- 172.21.15.110:0/1561123191 shutdown_connections 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 1 -- 172.21.15.110:0/1561123191 wait complete. 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.101+0000 7fce8db8b740 20 client.25661 trim_cache size 0 max 16384 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:12:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:12:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 -- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce700a2da0 msgr2=0x7fce700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce700a2da0 0x7fce700a5260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fce74098700 tx=0x7fce740978a0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 -- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce84103360 msgr2=0x7fce8419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce84103360 0x7fce8419eec0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fce7400eb10 tx=0x7fce7400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 -- 172.21.15.110:0/3039079448 shutdown_connections 2024-04-25T00:12:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fce700a2da0 0x7fce700a5260 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce84103360 0x7fce8419eec0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce841029a0 0x7fce84197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 --2- 172.21.15.110:0/3039079448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce841017b0 0x7fce84197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 -- 172.21.15.110:0/3039079448 >> 172.21.15.110:0/3039079448 conn(0x7fce840fcd00 msgr2=0x7fce84105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 -- 172.21.15.110:0/3039079448 shutdown_connections 2024-04-25T00:12:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:28.102+0000 7fce8adab640 1 -- 172.21.15.110:0/3039079448 wait complete. 2024-04-25T00:12:28.105 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:27 smithi173 ceph-mon[32402]: pgmap v663: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 856 op/s 2024-04-25T00:12:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:29 smithi188 ceph-mon[33285]: pgmap v664: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-04-25T00:12:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:29 smithi110 ceph-mon[29187]: pgmap v664: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-04-25T00:12:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:29 smithi173 ceph-mon[32402]: pgmap v664: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-04-25T00:12:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 565.0 s, 832.4 tps, lat 120.129 ms stddev 31.995 2024-04-25T00:12:32.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:31 smithi188 ceph-mon[33285]: pgmap v665: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-04-25T00:12:32.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:31 smithi110 ceph-mon[29187]: pgmap v665: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-04-25T00:12:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:31 smithi173 ceph-mon[32402]: pgmap v665: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-04-25T00:12:34.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:33 smithi188 ceph-mon[33285]: pgmap v666: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-04-25T00:12:34.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:33 smithi110 ceph-mon[29187]: pgmap v666: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-04-25T00:12:34.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:33 smithi173 ceph-mon[32402]: pgmap v666: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-04-25T00:12:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:35 smithi188 ceph-mon[33285]: pgmap v667: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 14 MiB/s wr, 979 op/s 2024-04-25T00:12:36.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:35 smithi110 ceph-mon[29187]: pgmap v667: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 14 MiB/s wr, 979 op/s 2024-04-25T00:12:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:35 smithi173 ceph-mon[32402]: pgmap v667: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 14 MiB/s wr, 979 op/s 2024-04-25T00:12:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 570.0 s, 850.2 tps, lat 117.283 ms stddev 15.940 2024-04-25T00:12:38.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:37 smithi188 ceph-mon[33285]: pgmap v668: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-04-25T00:12:38.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:37 smithi110 ceph-mon[29187]: pgmap v668: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-04-25T00:12:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:37 smithi173 ceph-mon[32402]: pgmap v668: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-04-25T00:12:40.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:39 smithi188 ceph-mon[33285]: pgmap v669: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-04-25T00:12:40.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:39 smithi110 ceph-mon[29187]: pgmap v669: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-04-25T00:12:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:39 smithi173 ceph-mon[32402]: pgmap v669: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-04-25T00:12:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.222+0000 7f4357dce640 1 -- 172.21.15.110:0/1608844813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4350101580 msgr2=0x7f4350107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.222+0000 7f4357dce640 1 --2- 172.21.15.110:0/1608844813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4350101580 0x7f4350107e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f434802b3e0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.223+0000 7f4357dce640 1 -- 172.21.15.110:0/1608844813 shutdown_connections 2024-04-25T00:12:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.223+0000 7f4357dce640 1 --2- 172.21.15.110:0/1608844813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4350101580 0x7f4350107e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.223+0000 7f4357dce640 1 --2- 172.21.15.110:0/1608844813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4350100bc0 0x7f4350101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.223+0000 7f4357dce640 1 --2- 172.21.15.110:0/1608844813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43500ff9d0 0x7f43500ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.223+0000 7f4357dce640 1 -- 172.21.15.110:0/1608844813 >> 172.21.15.110:0/1608844813 conn(0x7f43500faf60 msgr2=0x7f43500fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.223+0000 7f4357dce640 1 -- 172.21.15.110:0/1608844813 shutdown_connections 2024-04-25T00:12:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.223+0000 7f4357dce640 1 -- 172.21.15.110:0/1608844813 wait complete. 2024-04-25T00:12:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4357dce640 1 Processor -- start 2024-04-25T00:12:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4357dce640 1 -- start start 2024-04-25T00:12:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4357dce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43500ff9d0 0x7f43501a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4357dce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4350100bc0 0x7f43501a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4357dce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4350101580 0x7f43501a7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4357dce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f435006ba20 con 0x7f4350100bc0 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4357dce640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f435006b8a0 con 0x7f43500ff9d0 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4357dce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f435006bba0 con 0x7f4350101580 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f435636b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4350101580 0x7f43501a7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4355b6a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43500ff9d0 0x7f43501a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4355b6a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43500ff9d0 0x7f43501a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:58616/0 (socket says 172.21.15.110:58616) 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4355b6a640 1 -- 172.21.15.110:0/2146271886 learned_addr learned my addr 172.21.15.110:0/2146271886 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.224+0000 7f4355369640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4350100bc0 0x7f43501a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4355b6a640 1 -- 172.21.15.110:0/2146271886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4350101580 msgr2=0x7f43501a7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4355b6a640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4350101580 0x7f43501a7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4355b6a640 1 -- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4350100bc0 msgr2=0x7f43501a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4355b6a640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4350100bc0 0x7f43501a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4355b6a640 1 -- 172.21.15.110:0/2146271886 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4350101ee0 con 0x7f43500ff9d0 2024-04-25T00:12:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4355b6a640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43500ff9d0 0x7f43501a0700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f434c00c9a0 tx=0x7f434c00ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4346ffd640 1 -- 172.21.15.110:0/2146271886 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c004220 con 0x7f43500ff9d0 2024-04-25T00:12:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4346ffd640 1 -- 172.21.15.110:0/2146271886 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c0043c0 con 0x7f43500ff9d0 2024-04-25T00:12:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4346ffd640 1 -- 172.21.15.110:0/2146271886 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c004a00 con 0x7f43500ff9d0 2024-04-25T00:12:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4357dce640 1 -- 172.21.15.110:0/2146271886 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4350102170 con 0x7f43500ff9d0 2024-04-25T00:12:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.225+0000 7f4357dce640 1 -- 172.21.15.110:0/2146271886 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4350102670 con 0x7f43500ff9d0 2024-04-25T00:12:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.229+0000 7f4346ffd640 1 -- 172.21.15.110:0/2146271886 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f434c004ba0 con 0x7f43500ff9d0 2024-04-25T00:12:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.230+0000 7f4346ffd640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f433c0a2d30 0x7f433c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.230+0000 7f4346ffd640 1 -- 172.21.15.110:0/2146271886 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f434c0cf010 con 0x7f43500ff9d0 2024-04-25T00:12:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.230+0000 7f4355369640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f433c0a2d30 0x7f433c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.231+0000 7f4355369640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f433c0a2d30 0x7f433c0a51f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f43501a15b0 tx=0x7f434000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.231+0000 7f435abbb740 1 Processor -- start 2024-04-25T00:12:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.231+0000 7f435abbb740 1 -- start start 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.231+0000 7f435abbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652611e50 0x558652612270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f435abbb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652612840 0x558652617340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f435abbb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5586526179b0 0x558652619e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f435abbb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558652575cb0 con 0x558652612840 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f435abbb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55865251e5b0 con 0x5586526179b0 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f435abbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558652571c70 con 0x558652611e50 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355b6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652611e50 0x558652612270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f435636b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5586526179b0 0x558652619e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355b6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652611e50 0x558652612270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:60562/0 (socket says 172.21.15.110:60562) 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355b6a640 1 -- 172.21.15.110:0/3869962551 learned_addr learned my addr 172.21.15.110:0/3869962551 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355369640 1 --2- 172.21.15.110:0/3869962551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652612840 0x558652617340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355b6a640 1 -- 172.21.15.110:0/3869962551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5586526179b0 msgr2=0x558652619e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355b6a640 1 --2- 172.21.15.110:0/3869962551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5586526179b0 0x558652619e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355b6a640 1 -- 172.21.15.110:0/3869962551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652612840 msgr2=0x558652617340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355b6a640 1 --2- 172.21.15.110:0/3869962551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652612840 0x558652617340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.232+0000 7f4355b6a640 1 -- 172.21.15.110:0/3869962551 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865228a210 con 0x558652611e50 2024-04-25T00:12:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.233+0000 7f4355b6a640 1 --2- 172.21.15.110:0/3869962551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652611e50 0x558652612270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f434c09b5f0 tx=0x7f434c09b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5323afac4b37693e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.233+0000 7f433bfff640 1 -- 172.21.15.110:0/3869962551 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c01d040 con 0x558652611e50 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.233+0000 7f433bfff640 1 -- 172.21.15.110:0/3869962551 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c099bf0 con 0x558652611e50 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.233+0000 7f433bfff640 1 -- 172.21.15.110:0/3869962551 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c0a4020 con 0x558652611e50 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.234+0000 7f435abbb740 1 -- 172.21.15.110:0/3869962551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652611e50 msgr2=0x558652612270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.234+0000 7f435abbb740 1 --2- 172.21.15.110:0/3869962551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652611e50 0x558652612270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f434c09b5f0 tx=0x7f434c09b7f0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.234+0000 7f435abbb740 1 -- 172.21.15.110:0/3869962551 shutdown_connections 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.234+0000 7f435abbb740 1 --2- 172.21.15.110:0/3869962551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5586526179b0 0x558652619e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.234+0000 7f435abbb740 1 --2- 172.21.15.110:0/3869962551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652612840 0x558652617340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.234+0000 7f435abbb740 1 --2- 172.21.15.110:0/3869962551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652611e50 0x558652612270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.234+0000 7f435abbb740 1 -- 172.21.15.110:0/3869962551 >> 172.21.15.110:0/3869962551 conn(0x55865260d4c0 msgr2=0x55865260d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.235+0000 7f435abbb740 1 -- 172.21.15.110:0/3869962551 shutdown_connections 2024-04-25T00:12:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.235+0000 7f435abbb740 1 -- 172.21.15.110:0/3869962551 wait complete. 2024-04-25T00:12:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.235+0000 7f435abbb740 1 -- start start 2024-04-25T00:12:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.235+0000 7f435abbb740 1 Processor -- start 2024-04-25T00:12:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435abbb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x55865261c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435abbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x55865261c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435abbb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435abbb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55865261a960 con 0x5586526179b0 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435abbb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558652531b50 con 0x558652611e50 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435abbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55865261aae0 con 0x558652612840 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435abbb740 10 client.0 _ll_register_callbacks cb 0x55865256cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f4355369640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x55865261c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f4355369640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x55865261c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:60568/0 (socket says 172.21.15.110:60568) 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f4355369640 1 -- 172.21.15.110:0/3565993757 learned_addr learned my addr 172.21.15.110:0/3565993757 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f4355b6a640 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x55865261c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435636b640 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435636b640 1 -- 172.21.15.110:0/3565993757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 msgr2=0x55865261c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.236+0000 7f435636b640 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x55865261c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.237+0000 7f435636b640 1 -- 172.21.15.110:0/3565993757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 msgr2=0x55865261c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.237+0000 7f435636b640 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x55865261c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.237+0000 7f435636b640 1 -- 172.21.15.110:0/3565993757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558652356620 con 0x5586526179b0 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.237+0000 7f4355369640 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x55865261c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.237+0000 7f43397fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.237+0000 7f435636b640 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623850 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4348007780 tx=0x7f4348032560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a11f0fdd78316d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.237+0000 7f43397fa640 1 -- 172.21.15.110:0/3565993757 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348037040 con 0x5586526179b0 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.238+0000 7f43397fa640 1 -- 172.21.15.110:0/3565993757 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4348007c60 con 0x5586526179b0 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.238+0000 7f435abbb740 1 -- 172.21.15.110:0/3565993757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c099bf0 con 0x5586526179b0 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.238+0000 7f43397fa640 1 -- 172.21.15.110:0/3565993757 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4348037040 con 0x5586526179b0 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.238+0000 7f435abbb740 10 client.16089 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.238+0000 7f435abbb740 1 -- 172.21.15.110:0/3565993757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558652626c30 con 0x5586526179b0 2024-04-25T00:12:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f43397fa640 1 -- 172.21.15.110:0/3565993757 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c099bf0 con 0x5586526179b0 2024-04-25T00:12:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f43397fa640 1 -- 172.21.15.110:0/3565993757 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f4348046430 con 0x5586526179b0 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f435abbb740 10 client.16089 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f435abbb740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f435abbb740 20 client.16089 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f435abbb740 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5586526286e0 0x55865262ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f435abbb740 4 client.16089 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f435abbb740 1 -- 172.21.15.110:0/3565993757 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586525831f0 con 0x5586526286e0 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.239+0000 7f4355b6a640 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5586526286e0 0x55865262ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.240+0000 7f43397fa640 10 client.16089 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.240+0000 7f4355b6a640 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5586526286e0 0x55865262ab50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc71b847ea52f2f server_cookie=5888660b63a6c33c in_seq=0 out_seq=0 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.241+0000 7f43397fa640 1 -- 172.21.15.110:0/3565993757 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586525831f0 con 0x5586526286e0 2024-04-25T00:12:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.241+0000 7f43397fa640 10 client.16089 handle_command_reply: tid=0 2024-04-25T00:12:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.241+0000 7f435abbb740 1 client.16089 shutdown 2024-04-25T00:12:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.242+0000 7f435abbb740 1 -- 172.21.15.110:0/3565993757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 msgr2=0x558652623850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.242+0000 7f435abbb740 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623850 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4348007780 tx=0x7f4348032560 comp rx=0 tx=0).stop 2024-04-25T00:12:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.243+0000 7f435abbb740 1 -- 172.21.15.110:0/3565993757 shutdown_connections 2024-04-25T00:12:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.243+0000 7f435abbb740 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5586526286e0 0x55865262ab50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.243+0000 7f435abbb740 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623850 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.243+0000 7f435abbb740 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x55865261c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.243+0000 7f435abbb740 1 --2- 172.21.15.110:0/3565993757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x55865261c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.243+0000 7f435abbb740 1 -- 172.21.15.110:0/3565993757 >> 172.21.15.110:0/3565993757 conn(0x55865260d4c0 msgr2=0x55865260c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.244+0000 7f43397fa640 0 client.16089 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.244+0000 7f435abbb740 1 -- 172.21.15.110:0/3565993757 shutdown_connections 2024-04-25T00:12:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.244+0000 7f435abbb740 1 -- 172.21.15.110:0/3565993757 wait complete. 2024-04-25T00:12:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.244+0000 7f435abbb740 20 client.16089 trim_cache size 0 max 16384 2024-04-25T00:12:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.272+0000 7f435abbb740 1 Processor -- start 2024-04-25T00:12:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.272+0000 7f435abbb740 1 -- start start 2024-04-25T00:12:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.272+0000 7f435abbb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x558652613d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f435abbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x558652614240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355b6a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x558652613d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355b6a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x558652613d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:58642/0 (socket says 172.21.15.110:58642) 2024-04-25T00:12:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355b6a640 1 -- 172.21.15.110:0/46506938 learned_addr learned my addr 172.21.15.110:0/46506938 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355369640 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x558652614240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f435abbb740 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f435abbb740 1 -- 172.21.15.110:0/46506938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b5fb0 con 0x5586526179b0 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f435abbb740 1 -- 172.21.15.110:0/46506938 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b3880 con 0x558652611e50 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f435abbb740 1 -- 172.21.15.110:0/46506938 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b1670 con 0x558652612840 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f435636b640 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355b6a640 1 -- 172.21.15.110:0/46506938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 msgr2=0x558652614240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355b6a640 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x558652614240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355b6a640 1 -- 172.21.15.110:0/46506938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 msgr2=0x558652623810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355b6a640 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.273+0000 7f4355b6a640 1 -- 172.21.15.110:0/46506938 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586526082d0 con 0x558652611e50 2024-04-25T00:12:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.274+0000 7f4355b6a640 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x558652613d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f434c097d50 tx=0x7f434c097cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474931f60fbaa2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.274+0000 7f433b7fe640 1 -- 172.21.15.110:0/46506938 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c01d040 con 0x558652611e50 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.274+0000 7f433b7fe640 1 -- 172.21.15.110:0/46506938 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c0ccbf0 con 0x558652611e50 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.274+0000 7f433b7fe640 1 -- 172.21.15.110:0/46506938 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c0a4020 con 0x558652611e50 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.275+0000 7f435abbb740 1 -- 172.21.15.110:0/46506938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 msgr2=0x558652613d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.275+0000 7f435abbb740 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x558652613d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f434c097d50 tx=0x7f434c097cc0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.275+0000 7f435abbb740 1 -- 172.21.15.110:0/46506938 shutdown_connections 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.275+0000 7f435abbb740 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5586526179b0 0x558652623810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.275+0000 7f435abbb740 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558652612840 0x558652614240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.275+0000 7f435abbb740 1 --2- 172.21.15.110:0/46506938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652611e50 0x558652613d00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.275+0000 7f435abbb740 1 -- 172.21.15.110:0/46506938 >> 172.21.15.110:0/46506938 conn(0x55865260d4c0 msgr2=0x55865260d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.276+0000 7f435abbb740 1 -- 172.21.15.110:0/46506938 shutdown_connections 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.276+0000 7f435abbb740 1 -- 172.21.15.110:0/46506938 wait complete. 2024-04-25T00:12:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.276+0000 7f435abbb740 1 -- start start 2024-04-25T00:12:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.277+0000 7f435abbb740 1 Processor -- start 2024-04-25T00:12:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.277+0000 7f435abbb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652611e50 0x558652623820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.277+0000 7f435abbb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652612840 0x5586526202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.277+0000 7f435abbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586526179b0 0x5586526208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f435abbb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b1d40 con 0x558652611e50 2024-04-25T00:12:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355b6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652611e50 0x558652623820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f435abbb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b31b0 con 0x558652612840 2024-04-25T00:12:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f435abbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b2410 con 0x5586526179b0 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f435abbb740 10 client.0 _ll_register_callbacks cb 0x7f4348032400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355369640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652612840 0x5586526202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355369640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652612840 0x5586526202c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:58650/0 (socket says 172.21.15.110:58650) 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355b6a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652611e50 0x558652623820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60108/0 (socket says 172.21.15.110:60108) 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355369640 1 -- 172.21.15.110:0/2873682095 learned_addr learned my addr 172.21.15.110:0/2873682095 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355369640 1 -- 172.21.15.110:0/2873682095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586526179b0 msgr2=0x5586526208f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355369640 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586526179b0 0x5586526208f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355369640 1 -- 172.21.15.110:0/2873682095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652611e50 msgr2=0x558652623820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355369640 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652611e50 0x558652623820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.278+0000 7f4355369640 1 -- 172.21.15.110:0/2873682095 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865268cd40 con 0x558652612840 2024-04-25T00:12:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.279+0000 7f4355369640 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652612840 0x5586526202c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f434000b6e0 tx=0x7f4340004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127b4eb4f0ebb3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.279+0000 7f43397fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:12:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.279+0000 7f43397fa640 1 -- 172.21.15.110:0/2873682095 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434002e060 con 0x558652612840 2024-04-25T00:12:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.279+0000 7f43397fa640 1 -- 172.21.15.110:0/2873682095 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f43400293f0 con 0x558652612840 2024-04-25T00:12:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.279+0000 7f435abbb740 1 -- 172.21.15.110:0/2873682095 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c0ccbf0 con 0x558652612840 2024-04-25T00:12:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.280+0000 7f43397fa640 1 -- 172.21.15.110:0/2873682095 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f43400295f0 con 0x558652612840 2024-04-25T00:12:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.280+0000 7f435abbb740 10 client.25679 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.280+0000 7f435abbb740 1 -- 172.21.15.110:0/2873682095 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55865268e0c0 con 0x558652612840 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.280+0000 7f43397fa640 1 -- 172.21.15.110:0/2873682095 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c0ccbf0 con 0x558652612840 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.281+0000 7f43397fa640 1 -- 172.21.15.110:0/2873682095 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f4340028070 con 0x558652612840 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.281+0000 7f435abbb740 10 client.25679 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.281+0000 7f435abbb740 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.281+0000 7f435abbb740 20 client.25679 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.281+0000 7f435abbb740 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55865268ea20 0x558652690e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.282+0000 7f435abbb740 4 client.25679 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.282+0000 7f435abbb740 1 -- 172.21.15.110:0/2873682095 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55865257d540 con 0x55865268ea20 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.282+0000 7f4355b6a640 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55865268ea20 0x558652690e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.282+0000 7f4355b6a640 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55865268ea20 0x558652690e40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e08268bda50ea server_cookie=f6bb6a83b9b22561 in_seq=0 out_seq=0 2024-04-25T00:12:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.282+0000 7f43397fa640 10 client.25679 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.298+0000 7f43397fa640 1 -- 172.21.15.110:0/2873682095 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55865257d540 con 0x55865268ea20 2024-04-25T00:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.298+0000 7f43397fa640 10 client.25679 handle_command_reply: tid=0 2024-04-25T00:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.298+0000 7f435abbb740 1 client.25679 shutdown 2024-04-25T00:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.298+0000 7f435abbb740 1 -- 172.21.15.110:0/2873682095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652612840 msgr2=0x5586526202c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.298+0000 7f435abbb740 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652612840 0x5586526202c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f434000b6e0 tx=0x7f4340004730 comp rx=0 tx=0).stop 2024-04-25T00:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 1 -- 172.21.15.110:0/2873682095 shutdown_connections 2024-04-25T00:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55865268ea20 0x558652690e40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586526179b0 0x5586526208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558652612840 0x5586526202c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 1 --2- 172.21.15.110:0/2873682095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558652611e50 0x558652623820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 1 -- 172.21.15.110:0/2873682095 >> 172.21.15.110:0/2873682095 conn(0x55865260d4c0 msgr2=0x558652613d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f43397fa640 0 client.25679 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 1 -- 172.21.15.110:0/2873682095 shutdown_connections 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 1 -- 172.21.15.110:0/2873682095 wait complete. 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.299+0000 7f435abbb740 20 client.25679 trim_cache size 0 max 16384 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 -- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f433c0a2d30 msgr2=0x7f433c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f433c0a2d30 0x7f433c0a51f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f43501a15b0 tx=0x7f434000b390 comp rx=0 tx=0).stop 2024-04-25T00:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 -- 172.21.15.110:0/2146271886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43500ff9d0 msgr2=0x7f43501a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43500ff9d0 0x7f43501a0700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f434c00c9a0 tx=0x7f434c00ce60 comp rx=0 tx=0).stop 2024-04-25T00:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 -- 172.21.15.110:0/2146271886 shutdown_connections 2024-04-25T00:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f433c0a2d30 0x7f433c0a51f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4350101580 0x7f43501a7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4350100bc0 0x7f43501a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 --2- 172.21.15.110:0/2146271886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43500ff9d0 0x7f43501a0700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.300+0000 7f4357dce640 1 -- 172.21.15.110:0/2146271886 >> 172.21.15.110:0/2146271886 conn(0x7f43500faf60 msgr2=0x7f4350106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.301+0000 7f4357dce640 1 -- 172.21.15.110:0/2146271886 shutdown_connections 2024-04-25T00:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:41.301+0000 7f4357dce640 1 -- 172.21.15.110:0/2146271886 wait complete. 2024-04-25T00:12:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 575.0 s, 849.8 tps, lat 117.841 ms stddev 14.802 2024-04-25T00:12:42.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:41 smithi110 ceph-mon[29187]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-04-25T00:12:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:41 smithi173 ceph-mon[32402]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-04-25T00:12:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:41 smithi188 ceph-mon[33285]: pgmap v670: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 835 op/s 2024-04-25T00:12:44.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:43 smithi110 ceph-mon[29187]: pgmap v671: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-04-25T00:12:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:43 smithi173 ceph-mon[32402]: pgmap v671: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-04-25T00:12:44.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:43 smithi188 ceph-mon[33285]: pgmap v671: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-04-25T00:12:45.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:44 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:45.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:44 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:45.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:44 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:46.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:45 smithi110 ceph-mon[29187]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 862 op/s 2024-04-25T00:12:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:45 smithi173 ceph-mon[32402]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 862 op/s 2024-04-25T00:12:46.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:45 smithi188 ceph-mon[33285]: pgmap v672: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 862 op/s 2024-04-25T00:12:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 580.0 s, 782.6 tps, lat 126.309 ms stddev 32.802 2024-04-25T00:12:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:47 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:47.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:47 smithi173 ceph-mon[32402]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-04-25T00:12:47.902 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:47.902 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:47 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:47.902 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:47 smithi188 ceph-mon[33285]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-04-25T00:12:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:47 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:48.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:47 smithi110 ceph-mon[29187]: pgmap v673: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-04-25T00:12:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:48 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:48 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:48 smithi188 ceph-mon[33285]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-04-25T00:12:49.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:48 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:49.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:48 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:49.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:48 smithi110 ceph-mon[29187]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-04-25T00:12:49.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:48 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:49.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:48 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:49.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:48 smithi173 ceph-mon[32402]: pgmap v674: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-04-25T00:12:50.187 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:50.188 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:50.188 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:50.188 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:50.188 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:49 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:50.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:50.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:50.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:50.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:50.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:49 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:50.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:50.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:50.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:50.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:50.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:49 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:12:51.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:50 smithi188 ceph-mon[33285]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-04-25T00:12:51.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:50 smithi110 ceph-mon[29187]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-04-25T00:12:51.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:50 smithi173 ceph-mon[32402]: pgmap v675: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-04-25T00:12:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 585.0 s, 774.4 tps, lat 130.645 ms stddev 35.897 2024-04-25T00:12:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:53 smithi188 ceph-mon[33285]: pgmap v676: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-04-25T00:12:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:53 smithi110 ceph-mon[29187]: pgmap v676: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-04-25T00:12:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:53 smithi173 ceph-mon[32402]: pgmap v676: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-04-25T00:12:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/961984732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e381039c0 msgr2=0x7f3e38103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/961984732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e381039c0 0x7f3e38103de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009860 tx=0x7f3e2c02a340 comp rx=0 tx=0).stop 2024-04-25T00:12:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/961984732 shutdown_connections 2024-04-25T00:12:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/961984732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e38105570 0x7f3e3810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/961984732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e38104bb0 0x7f3e38105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/961984732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e381039c0 0x7f3e38103de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/961984732 >> 172.21.15.110:0/961984732 conn(0x7f3e380fef30 msgr2=0x7f3e38101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/961984732 shutdown_connections 2024-04-25T00:12:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.429+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/961984732 wait complete. 2024-04-25T00:12:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.430+0000 7f3e3d3fb640 1 Processor -- start 2024-04-25T00:12:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.430+0000 7f3e3d3fb640 1 -- start start 2024-04-25T00:12:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.430+0000 7f3e3d3fb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e381039c0 0x7f3e38199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.430+0000 7f3e3d3fb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e38104bb0 0x7f3e3819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e36ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e381039c0 0x7f3e38199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e3d3fb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e38105570 0x7f3e381a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e36ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e381039c0 0x7f3e38199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54666/0 (socket says 172.21.15.110:54666) 2024-04-25T00:12:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e36ffd640 1 -- 172.21.15.110:0/3369648369 learned_addr learned my addr 172.21.15.110:0/3369648369 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3810f4e0 con 0x7f3e381039c0 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e367fc640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e38104bb0 0x7f3e3819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3810f360 con 0x7f3e38105570 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3810f660 con 0x7f3e38104bb0 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e38105570 0x7f3e381a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e377fe640 1 -- 172.21.15.110:0/3369648369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e38104bb0 msgr2=0x7f3e3819a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e38104bb0 0x7f3e3819a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.431+0000 7f3e377fe640 1 -- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e381039c0 msgr2=0x7f3e38199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.432+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e381039c0 0x7f3e38199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.432+0000 7f3e377fe640 1 -- 172.21.15.110:0/3369648369 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e381a1780 con 0x7f3e38105570 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.432+0000 7f3e367fc640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e38104bb0 0x7f3e3819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.432+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e381039c0 0x7f3e38199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.432+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e38105570 0x7f3e381a1110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000ef00 tx=0x7f3e3000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.432+0000 7f3e1ffff640 1 -- 172.21.15.110:0/3369648369 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30004b20 con 0x7f3e38105570 2024-04-25T00:12:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.432+0000 7f3e1ffff640 1 -- 172.21.15.110:0/3369648369 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e30003e20 con 0x7f3e38105570 2024-04-25T00:12:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.432+0000 7f3e1ffff640 1 -- 172.21.15.110:0/3369648369 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000b570 con 0x7f3e38105570 2024-04-25T00:12:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.433+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e381a1a70 con 0x7f3e38105570 2024-04-25T00:12:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.433+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e381a1fc0 con 0x7f3e38105570 2024-04-25T00:12:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.435+0000 7f3e1ffff640 1 -- 172.21.15.110:0/3369648369 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3e3001d020 con 0x7f3e38105570 2024-04-25T00:12:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.435+0000 7f3e401df740 1 Processor -- start 2024-04-25T00:12:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e401df740 1 -- start start 2024-04-25T00:12:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e1ffff640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e200a2d30 0x7f3e200a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e401df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d92e50 0x556137d93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e401df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d93840 0x556137d98340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e401df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d989b0 0x556137d9ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e401df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556137cf6cb0 con 0x556137d989b0 2024-04-25T00:12:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e401df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556137c9f5b0 con 0x556137d93840 2024-04-25T00:12:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e401df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556137cf2c70 con 0x556137d92e50 2024-04-25T00:12:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e1ffff640 1 -- 172.21.15.110:0/3369648369 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3e300cee80 con 0x7f3e38105570 2024-04-25T00:12:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e36ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d92e50 0x556137d93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e377fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d989b0 0x556137d9ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e36ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d92e50 0x556137d93270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:42252/0 (socket says 172.21.15.110:42252) 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e36ffd640 1 -- 172.21.15.110:0/3462559369 learned_addr learned my addr 172.21.15.110:0/3462559369 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e367fc640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e200a2d30 0x7f3e200a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e36ffd640 1 -- 172.21.15.110:0/3462559369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d93840 msgr2=0x556137d98340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/3462559369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d93840 0x556137d98340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.436+0000 7f3e36ffd640 1 -- 172.21.15.110:0/3462559369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d989b0 msgr2=0x556137d9ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.437+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/3462559369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d989b0 0x556137d9ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.437+0000 7f3e36ffd640 1 -- 172.21.15.110:0/3462559369 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556137a0b210 con 0x556137d92e50 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.437+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3462559369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d989b0 0x556137d9ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.437+0000 7f3e367fc640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e200a2d30 0x7f3e200a51f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3e3819b070 tx=0x7f3e28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.437+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/3462559369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d92e50 0x556137d93270 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c035040 tx=0x7f3e2c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c391fd063cb08ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.437+0000 7f3e1d7fa640 1 -- 172.21.15.110:0/3462559369 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c037040 con 0x556137d92e50 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.437+0000 7f3e1d7fa640 1 -- 172.21.15.110:0/3462559369 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e2c038040 con 0x556137d92e50 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.437+0000 7f3e1d7fa640 1 -- 172.21.15.110:0/3462559369 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c032bb0 con 0x556137d92e50 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.438+0000 7f3e401df740 1 -- 172.21.15.110:0/3462559369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d92e50 msgr2=0x556137d93270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.438+0000 7f3e401df740 1 --2- 172.21.15.110:0/3462559369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d92e50 0x556137d93270 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c035040 tx=0x7f3e2c009510 comp rx=0 tx=0).stop 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.438+0000 7f3e401df740 1 -- 172.21.15.110:0/3462559369 shutdown_connections 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.438+0000 7f3e401df740 1 --2- 172.21.15.110:0/3462559369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d989b0 0x556137d9ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.438+0000 7f3e401df740 1 --2- 172.21.15.110:0/3462559369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d93840 0x556137d98340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.438+0000 7f3e401df740 1 --2- 172.21.15.110:0/3462559369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d92e50 0x556137d93270 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.438+0000 7f3e401df740 1 -- 172.21.15.110:0/3462559369 >> 172.21.15.110:0/3462559369 conn(0x556137d8e4c0 msgr2=0x556137d8e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 -- 172.21.15.110:0/3462559369 shutdown_connections 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 -- 172.21.15.110:0/3462559369 wait complete. 2024-04-25T00:12:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 -- start start 2024-04-25T00:12:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 Processor -- start 2024-04-25T00:12:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d9d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d9d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137da4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556137d9b900 con 0x556137d93840 2024-04-25T00:12:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556137cb2b50 con 0x556137d92e50 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556137d9ba80 con 0x556137d989b0 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e401df740 10 client.0 _ll_register_callbacks cb 0x556137cedca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e36ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d9d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e367fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d9d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e36ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d9d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:36476/0 (socket says 172.21.15.110:36476) 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.439+0000 7f3e36ffd640 1 -- 172.21.15.110:0/164546049 learned_addr learned my addr 172.21.15.110:0/164546049 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e377fe640 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137da4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e367fc640 1 -- 172.21.15.110:0/164546049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 msgr2=0x556137da4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e367fc640 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137da4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e367fc640 1 -- 172.21.15.110:0/164546049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 msgr2=0x556137d9d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e367fc640 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d9d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e367fc640 1 -- 172.21.15.110:0/164546049 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556137ad7620 con 0x556137d93840 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d9d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e367fc640 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d9d810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x556137d942b0 tx=0x7f3e28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30e7422079ef49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e1aff5640 1 -- 172.21.15.110:0/164546049 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800f9d0 con 0x556137d93840 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e1aff5640 1 -- 172.21.15.110:0/164546049 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3e2800fb70 con 0x556137d93840 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.440+0000 7f3e401df740 1 -- 172.21.15.110:0/164546049 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e2c038040 con 0x556137d93840 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e1aff5640 1 -- 172.21.15.110:0/164546049 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e2803f070 con 0x556137d93840 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e401df740 10 client.16107 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e401df740 1 -- 172.21.15.110:0/164546049 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556137da7c90 con 0x556137d93840 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e1aff5640 1 -- 172.21.15.110:0/164546049 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e2c038040 con 0x556137d93840 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e1aff5640 1 -- 172.21.15.110:0/164546049 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f3e2801a030 con 0x556137d93840 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e401df740 10 client.16107 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e401df740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e401df740 20 client.16107 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e401df740 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556137da9360 0x556137dab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e401df740 4 client.16107 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e401df740 1 -- 172.21.15.110:0/164546049 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556137d041f0 con 0x556137da9360 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.441+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556137da9360 0x556137dab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.442+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556137da9360 0x556137dab7b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89d7709d17205fc server_cookie=32acd9651937442 in_seq=0 out_seq=0 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.442+0000 7f3e1aff5640 10 client.16107 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e1aff5640 1 -- 172.21.15.110:0/164546049 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556137d041f0 con 0x556137da9360 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e1aff5640 10 client.16107 handle_command_reply: tid=0 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 client.16107 shutdown 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 -- 172.21.15.110:0/164546049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 msgr2=0x556137d9d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d9d810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x556137d942b0 tx=0x7f3e28004730 comp rx=0 tx=0).stop 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 -- 172.21.15.110:0/164546049 shutdown_connections 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556137da9360 0x556137dab7b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e1aff5640 0 client.16107 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137da4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d9d810 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 --2- 172.21.15.110:0/164546049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d9d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 -- 172.21.15.110:0/164546049 >> 172.21.15.110:0/164546049 conn(0x556137d8e4c0 msgr2=0x556137d8d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 -- 172.21.15.110:0/164546049 shutdown_connections 2024-04-25T00:12:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 1 -- 172.21.15.110:0/164546049 wait complete. 2024-04-25T00:12:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.443+0000 7f3e401df740 20 client.16107 trim_cache size 0 max 16384 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e401df740 1 Processor -- start 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e401df740 1 -- start start 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e401df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e401df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d95240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e401df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137da4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e401df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381a9100 con 0x556137d93840 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e401df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381a69d0 con 0x556137d92e50 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e401df740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381a4820 con 0x556137d989b0 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e367fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d95240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e367fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d95240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54694/0 (socket says 172.21.15.110:54694) 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e367fc640 1 -- 172.21.15.110:0/1845448837 learned_addr learned my addr 172.21.15.110:0/1845448837 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e377fe640 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137da4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.464+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e367fc640 1 -- 172.21.15.110:0/1845448837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 msgr2=0x556137da4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e367fc640 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137da4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e367fc640 1 -- 172.21.15.110:0/1845448837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 msgr2=0x556137d94d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e367fc640 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d94d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e367fc640 1 -- 172.21.15.110:0/1845448837 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556137d93c20 con 0x556137d93840 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e367fc640 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d95240 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3e280059f0 tx=0x7f3e28011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcdc2119bcbcf14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e1cff9640 1 -- 172.21.15.110:0/1845448837 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800f9d0 con 0x556137d93840 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e1cff9640 1 -- 172.21.15.110:0/1845448837 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e28048070 con 0x556137d93840 2024-04-25T00:12:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.465+0000 7f3e1cff9640 1 -- 172.21.15.110:0/1845448837 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28038470 con 0x556137d93840 2024-04-25T00:12:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 -- 172.21.15.110:0/1845448837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 msgr2=0x556137d95240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d95240 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3e280059f0 tx=0x7f3e28011dc0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 -- 172.21.15.110:0/1845448837 shutdown_connections 2024-04-25T00:12:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137da4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137d95240 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 --2- 172.21.15.110:0/1845448837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137d94d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 -- 172.21.15.110:0/1845448837 >> 172.21.15.110:0/1845448837 conn(0x556137d8e4c0 msgr2=0x556137d8e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 -- 172.21.15.110:0/1845448837 shutdown_connections 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 -- 172.21.15.110:0/1845448837 wait complete. 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 -- start start 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.466+0000 7f3e401df740 1 Processor -- start 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e401df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137da1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e401df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137da1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e401df740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137e0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e367fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137da1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e401df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381a4e90 con 0x556137d93840 2024-04-25T00:12:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e367fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137da1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54708/0 (socket says 172.21.15.110:54708) 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e367fc640 1 -- 172.21.15.110:0/3360003842 learned_addr learned my addr 172.21.15.110:0/3360003842 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137da1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381a6300 con 0x556137d92e50 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381a5560 con 0x556137d989b0 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137e0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.467+0000 7f3e401df740 10 client.0 _ll_register_callbacks cb 0x7f3e28004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e377fe640 1 -- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 msgr2=0x556137da1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137da1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e377fe640 1 -- 172.21.15.110:0/3360003842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 msgr2=0x556137da1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137da1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e377fe640 1 -- 172.21.15.110:0/3360003842 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556137e0de60 con 0x556137d989b0 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e367fc640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137da1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137da1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e377fe640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137e0d920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000cc40 tx=0x7f3e30004910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c572c880d07fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e1aff5640 1 -- 172.21.15.110:0/3360003842 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3001a040 con 0x556137d989b0 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e1aff5640 1 -- 172.21.15.110:0/3360003842 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3e300520f0 con 0x556137d989b0 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.468+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e28048070 con 0x556137d989b0 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e1aff5640 1 -- 172.21.15.110:0/3360003842 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e300a4320 con 0x556137d989b0 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e401df740 10 client.25675 fetch_fsmap learned FSMap version 32 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556137e0f160 con 0x556137d989b0 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e1aff5640 1 -- 172.21.15.110:0/3360003842 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e28048070 con 0x556137d989b0 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e1aff5640 1 -- 172.21.15.110:0/3360003842 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f3e300bb040 con 0x556137d989b0 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e401df740 10 client.25675 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e401df740 10 client.25675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e401df740 20 client.25675 populate_metadata read hostname 'smithi110' 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e401df740 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556137e0fab0 0x556137e11ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e401df740 4 client.25675 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:12:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556137cfe540 con 0x556137e0fab0 2024-04-25T00:12:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.469+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556137e0fab0 0x556137e11ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.470+0000 7f3e1aff5640 10 client.25675 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.470+0000 7f3e36ffd640 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556137e0fab0 0x556137e11ed0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faef0db8dd243b0 server_cookie=da6388358f7c3077 in_seq=0 out_seq=0 2024-04-25T00:12:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.484+0000 7f3e1aff5640 1 -- 172.21.15.110:0/3360003842 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556137cfe540 con 0x556137e0fab0 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.484+0000 7f3e1aff5640 10 client.25675 handle_command_reply: tid=0 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.484+0000 7f3e401df740 1 client.25675 shutdown 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 msgr2=0x556137e0d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137e0d920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000cc40 tx=0x7f3e30004910 comp rx=0 tx=0).stop 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 shutdown_connections 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556137e0fab0 0x556137e11ed0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556137d989b0 0x556137e0d920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556137d93840 0x556137da1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 --2- 172.21.15.110:0/3360003842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556137d92e50 0x556137da1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 >> 172.21.15.110:0/3360003842 conn(0x556137d8e4c0 msgr2=0x556137d9d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e1aff5640 0 client.25675 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:12:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 shutdown_connections 2024-04-25T00:12:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 1 -- 172.21.15.110:0/3360003842 wait complete. 2024-04-25T00:12:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.485+0000 7f3e401df740 20 client.25675 trim_cache size 0 max 16384 2024-04-25T00:12:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:12:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:12:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:12:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:12:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e200a2d30 msgr2=0x7f3e200a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e200a2d30 0x7f3e200a51f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3e3819b070 tx=0x7f3e28005f90 comp rx=0 tx=0).stop 2024-04-25T00:12:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e38105570 msgr2=0x7f3e381a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e38105570 0x7f3e381a1110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000ef00 tx=0x7f3e3000c740 comp rx=0 tx=0).stop 2024-04-25T00:12:54.488 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 shutdown_connections 2024-04-25T00:12:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3e200a2d30 0x7f3e200a51f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e38105570 0x7f3e381a1110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e38104bb0 0x7f3e3819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 --2- 172.21.15.110:0/3369648369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e381039c0 0x7f3e38199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 >> 172.21.15.110:0/3369648369 conn(0x7f3e380fef30 msgr2=0x7f3e38107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 shutdown_connections 2024-04-25T00:12:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:12:54.486+0000 7f3e3d3fb640 1 -- 172.21.15.110:0/3369648369 wait complete. 2024-04-25T00:12:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:55 smithi188 ceph-mon[33285]: pgmap v677: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-04-25T00:12:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:55 smithi110 ceph-mon[29187]: pgmap v677: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-04-25T00:12:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:55 smithi173 ceph-mon[32402]: pgmap v677: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-04-25T00:12:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 590.0 s, 799.2 tps, lat 125.009 ms stddev 23.899 2024-04-25T00:12:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:57 smithi110 ceph-mon[29187]: pgmap v678: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.7 MiB/s wr, 765 op/s 2024-04-25T00:12:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:57 smithi173 ceph-mon[32402]: pgmap v678: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.7 MiB/s wr, 765 op/s 2024-04-25T00:12:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:57 smithi188 ceph-mon[33285]: pgmap v678: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.7 MiB/s wr, 765 op/s 2024-04-25T00:13:00.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:12:59 smithi110 ceph-mon[29187]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-04-25T00:13:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:12:59 smithi173 ceph-mon[32402]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-04-25T00:13:00.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:12:59 smithi188 ceph-mon[33285]: pgmap v679: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-04-25T00:13:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 595.0 s, 836.8 tps, lat 119.616 ms stddev 14.415 2024-04-25T00:13:02.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:01 smithi110 ceph-mon[29187]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 877 op/s 2024-04-25T00:13:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:01 smithi173 ceph-mon[32402]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 877 op/s 2024-04-25T00:13:02.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:01 smithi188 ceph-mon[33285]: pgmap v680: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 877 op/s 2024-04-25T00:13:04.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:03 smithi110 ceph-mon[29187]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:13:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:03 smithi173 ceph-mon[32402]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:13:04.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:03 smithi188 ceph-mon[33285]: pgmap v681: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:13:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:05 smithi110 ceph-mon[29187]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 973 op/s 2024-04-25T00:13:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:05 smithi173 ceph-mon[32402]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 973 op/s 2024-04-25T00:13:06.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:05 smithi188 ceph-mon[33285]: pgmap v682: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 973 op/s 2024-04-25T00:13:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 600.0 s, 890.2 tps, lat 112.419 ms stddev 16.304 2024-04-25T00:13:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.619+0000 7fc96c922640 1 -- 172.21.15.110:0/2387606642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968100030 msgr2=0x7fc968100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.619+0000 7fc96c922640 1 --2- 172.21.15.110:0/2387606642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968100030 0x7fc968100450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc958009900 tx=0x7fc95802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.620+0000 7fc96c922640 1 -- 172.21.15.110:0/2387606642 shutdown_connections 2024-04-25T00:13:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.620+0000 7fc96c922640 1 --2- 172.21.15.110:0/2387606642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc968101330 0x7fc9681017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.620+0000 7fc96c922640 1 --2- 172.21.15.110:0/2387606642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc968100990 0x7fc968100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.620+0000 7fc96c922640 1 --2- 172.21.15.110:0/2387606642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968100030 0x7fc968100450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.620+0000 7fc96c922640 1 -- 172.21.15.110:0/2387606642 >> 172.21.15.110:0/2387606642 conn(0x7fc9680fbc10 msgr2=0x7fc9680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.621+0000 7fc96c922640 1 -- 172.21.15.110:0/2387606642 shutdown_connections 2024-04-25T00:13:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.621+0000 7fc96c922640 1 -- 172.21.15.110:0/2387606642 wait complete. 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.622+0000 7fc96c922640 1 Processor -- start 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.622+0000 7fc96c922640 1 -- start start 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.622+0000 7fc96c922640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc968100030 0x7fc968199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.622+0000 7fc96c922640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc968100990 0x7fc96819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.622+0000 7fc96c922640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968101330 0x7fc9681a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc965d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc968100990 0x7fc96819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc96659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc968100030 0x7fc968199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc966d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968101330 0x7fc9681a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc96659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc968100030 0x7fc968199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:45302/0 (socket says 172.21.15.110:45302) 2024-04-25T00:13:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc96659c640 1 -- 172.21.15.110:0/1533677209 learned_addr learned my addr 172.21.15.110:0/1533677209 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc96810f510 con 0x7fc968100990 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc96810f390 con 0x7fc968101330 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc96810f690 con 0x7fc968100030 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc966d9d640 1 -- 172.21.15.110:0/1533677209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc968100030 msgr2=0x7fc968199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc966d9d640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc968100030 0x7fc968199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc966d9d640 1 -- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc968100990 msgr2=0x7fc96819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc966d9d640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc968100990 0x7fc96819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc966d9d640 1 -- 172.21.15.110:0/1533677209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9681a17e0 con 0x7fc968101330 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc965d9b640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc968100990 0x7fc96819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.623+0000 7fc966d9d640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968101330 0x7fc9681a1170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc96000e940 tx=0x7fc96000ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.624+0000 7fc9577fe640 1 -- 172.21.15.110:0/1533677209 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960004d20 con 0x7fc968101330 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.624+0000 7fc9577fe640 1 -- 172.21.15.110:0/1533677209 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc960005020 con 0x7fc968101330 2024-04-25T00:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.624+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9681a1ad0 con 0x7fc968101330 2024-04-25T00:13:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.624+0000 7fc9577fe640 1 -- 172.21.15.110:0/1533677209 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9600116d0 con 0x7fc968101330 2024-04-25T00:13:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.624+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9681a2020 con 0x7fc968101330 2024-04-25T00:13:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.626+0000 7fc9577fe640 1 -- 172.21.15.110:0/1533677209 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc96000ca60 con 0x7fc968101330 2024-04-25T00:13:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.627+0000 7fc96f633740 1 Processor -- start 2024-04-25T00:13:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.627+0000 7fc9577fe640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9500a2d30 0x7fc9500a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.627+0000 7fc9577fe640 1 -- 172.21.15.110:0/1533677209 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc9600cf090 con 0x7fc968101330 2024-04-25T00:13:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96f633740 1 -- start start 2024-04-25T00:13:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96f633740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac20b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc965d9b640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9500a2d30 0x7fc9500a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96f633740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac210340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96f633740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac212e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96f633740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618ac16ecb0 con 0x5618ac2109b0 2024-04-25T00:13:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96f633740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5618ac1175b0 con 0x5618ac20b840 2024-04-25T00:13:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96f633740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618ac16ac70 con 0x5618ac20ae50 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac20b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac20b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:45306/0 (socket says 172.21.15.110:45306) 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc965d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac210340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96659c640 1 -- 172.21.15.110:0/2974474627 learned_addr learned my addr 172.21.15.110:0/2974474627 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc966d9d640 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac212e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96659c640 1 -- 172.21.15.110:0/2974474627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 msgr2=0x5618ac210340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96659c640 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac210340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96659c640 1 -- 172.21.15.110:0/2974474627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 msgr2=0x5618ac212e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96659c640 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac212e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc96659c640 1 -- 172.21.15.110:0/2974474627 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618abe83210 con 0x5618ac20ae50 2024-04-25T00:13:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.628+0000 7fc966d9d640 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac212e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.629+0000 7fc96659c640 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac20b270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc958005880 tx=0x7fc958031590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1041e3de6aa78f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.629+0000 7fc965d9b640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9500a2d30 0x7fc9500a51f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc96819b0d0 tx=0x7fc95c015530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.629+0000 7fc954ff9640 1 -- 172.21.15.110:0/2974474627 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958035020 con 0x5618ac20ae50 2024-04-25T00:13:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.629+0000 7fc954ff9640 1 -- 172.21.15.110:0/2974474627 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc958041070 con 0x5618ac20ae50 2024-04-25T00:13:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.629+0000 7fc954ff9640 1 -- 172.21.15.110:0/2974474627 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580046c0 con 0x5618ac20ae50 2024-04-25T00:13:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 -- 172.21.15.110:0/2974474627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 msgr2=0x5618ac20b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac20b270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc958005880 tx=0x7fc958031590 comp rx=0 tx=0).stop 2024-04-25T00:13:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 -- 172.21.15.110:0/2974474627 shutdown_connections 2024-04-25T00:13:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac212e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac210340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 --2- 172.21.15.110:0/2974474627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac20b270 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 -- 172.21.15.110:0/2974474627 >> 172.21.15.110:0/2974474627 conn(0x5618ac2064c0 msgr2=0x5618ac2068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 -- 172.21.15.110:0/2974474627 shutdown_connections 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 -- 172.21.15.110:0/2974474627 wait complete. 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.630+0000 7fc96f633740 1 -- start start 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96f633740 1 Processor -- start 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96f633740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac215270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96f633740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac2157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96f633740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac21c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96f633740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618ac213900 con 0x5618ac2109b0 2024-04-25T00:13:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96f633740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5618ac12ab50 con 0x5618ac20b840 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96f633740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618ac213a80 con 0x5618ac20ae50 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96f633740 10 client.0 _ll_register_callbacks cb 0x5618ac165ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc965d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac2157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac215270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc965d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac2157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:53438/0 (socket says 172.21.15.110:53438) 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc965d9b640 1 -- 172.21.15.110:0/1874463307 learned_addr learned my addr 172.21.15.110:0/1874463307 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc966d9d640 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac21c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96659c640 1 -- 172.21.15.110:0/1874463307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 msgr2=0x5618ac2157b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96659c640 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac2157b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96659c640 1 -- 172.21.15.110:0/1874463307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 msgr2=0x5618ac21c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96659c640 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac21c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc96659c640 1 -- 172.21.15.110:0/1874463307 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618abf4f620 con 0x5618ac20ae50 2024-04-25T00:13:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.631+0000 7fc965d9b640 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac2157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.632+0000 7fc96659c640 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac215270 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc958005d90 tx=0x7fc958031a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c94cc5417924e868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.632+0000 7fc94dffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.632+0000 7fc94dffb640 1 -- 172.21.15.110:0/1874463307 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958054060 con 0x5618ac20ae50 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.632+0000 7fc94dffb640 1 -- 172.21.15.110:0/1874463307 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc958004ad0 con 0x5618ac20ae50 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.632+0000 7fc96f633740 1 -- 172.21.15.110:0/1874463307 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc958041070 con 0x5618ac20ae50 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.632+0000 7fc94dffb640 1 -- 172.21.15.110:0/1874463307 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc958035040 con 0x5618ac20ae50 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc96f633740 10 client.25687 fetch_fsmap learned FSMap version 32 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc96f633740 1 -- 172.21.15.110:0/1874463307 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618ac213900 con 0x5618ac20ae50 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc94dffb640 1 -- 172.21.15.110:0/1874463307 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc958041070 con 0x5618ac20ae50 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc94dffb640 1 -- 172.21.15.110:0/1874463307 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc95803a040 con 0x5618ac20ae50 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc96f633740 10 client.25687 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc96f633740 10 client.25687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc96f633740 20 client.25687 populate_metadata read hostname 'smithi110' 2024-04-25T00:13:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc96f633740 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618ac220df0 0x5618ac2231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc96f633740 4 client.25687 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:13:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.633+0000 7fc96f633740 1 -- 172.21.15.110:0/1874463307 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618ac17c1f0 con 0x5618ac220df0 2024-04-25T00:13:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.634+0000 7fc965d9b640 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618ac220df0 0x5618ac2231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.634+0000 7fc965d9b640 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618ac220df0 0x5618ac2231e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473274c4e657df93 server_cookie=34bb4ef7178df1c9 in_seq=0 out_seq=0 2024-04-25T00:13:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.634+0000 7fc94dffb640 10 client.25687 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc94dffb640 1 -- 172.21.15.110:0/1874463307 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618ac17c1f0 con 0x5618ac220df0 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc94dffb640 10 client.25687 handle_command_reply: tid=0 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 client.25687 shutdown 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 -- 172.21.15.110:0/1874463307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 msgr2=0x5618ac215270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac215270 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc958005d90 tx=0x7fc958031a40 comp rx=0 tx=0).stop 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 -- 172.21.15.110:0/1874463307 shutdown_connections 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618ac220df0 0x5618ac2231e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc94dffb640 0 client.25687 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac21c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac2157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 --2- 172.21.15.110:0/1874463307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac215270 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 -- 172.21.15.110:0/1874463307 >> 172.21.15.110:0/1874463307 conn(0x5618ac2064c0 msgr2=0x5618ac2055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 -- 172.21.15.110:0/1874463307 shutdown_connections 2024-04-25T00:13:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 1 -- 172.21.15.110:0/1874463307 wait complete. 2024-04-25T00:13:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.635+0000 7fc96f633740 20 client.25687 trim_cache size 0 max 16384 2024-04-25T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.656+0000 7fc96f633740 1 Processor -- start 2024-04-25T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.656+0000 7fc96f633740 1 -- start start 2024-04-25T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc96f633740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac204d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc96f633740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac205270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc96f633740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac2057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc96f633740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9681a91c0 con 0x5618ac2109b0 2024-04-25T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc96f633740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9681a6a90 con 0x5618ac20b840 2024-04-25T00:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc96f633740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9681a48e0 con 0x5618ac20ae50 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac2057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc965d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac205270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac2057b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36100/0 (socket says 172.21.15.110:36100) 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 -- 172.21.15.110:0/2589077785 learned_addr learned my addr 172.21.15.110:0/2589077785 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc96659c640 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac204d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 -- 172.21.15.110:0/2589077785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 msgr2=0x5618ac204d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac204d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 -- 172.21.15.110:0/2589077785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 msgr2=0x5618ac205270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac205270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 -- 172.21.15.110:0/2589077785 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618ac20bc20 con 0x5618ac2109b0 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc965d9b640 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac205270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc966d9d640 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac2057b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc960005880 tx=0x7fc96009b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae5800536da2f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.657+0000 7fc94ffff640 1 -- 172.21.15.110:0/2589077785 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96001f040 con 0x5618ac2109b0 2024-04-25T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc94ffff640 1 -- 172.21.15.110:0/2589077785 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc960084e50 con 0x5618ac2109b0 2024-04-25T00:13:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc94ffff640 1 -- 172.21.15.110:0/2589077785 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9600051d0 con 0x5618ac2109b0 2024-04-25T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 -- 172.21.15.110:0/2589077785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 msgr2=0x5618ac2057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac2057b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc960005880 tx=0x7fc96009b220 comp rx=0 tx=0).stop 2024-04-25T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 -- 172.21.15.110:0/2589077785 shutdown_connections 2024-04-25T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac2109b0 0x5618ac2057b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac20b840 0x5618ac205270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 --2- 172.21.15.110:0/2589077785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20ae50 0x5618ac204d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 -- 172.21.15.110:0/2589077785 >> 172.21.15.110:0/2589077785 conn(0x5618ac2064c0 msgr2=0x5618ac2068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 -- 172.21.15.110:0/2589077785 shutdown_connections 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.658+0000 7fc96f633740 1 -- 172.21.15.110:0/2589077785 wait complete. 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 1 -- start start 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 1 Processor -- start 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac20ae50 0x5618ac218a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20b840 0x5618ac218f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac2109b0 0x5618ac2852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9681a4f50 con 0x5618ac20ae50 2024-04-25T00:13:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9681a63c0 con 0x5618ac2109b0 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9681a5620 con 0x5618ac20b840 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc96f633740 10 client.0 _ll_register_callbacks cb 0x7fc95c015400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.659+0000 7fc965d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20b840 0x5618ac218f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac2109b0 0x5618ac2852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac2109b0 0x5618ac2852e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:53452/0 (socket says 172.21.15.110:53452) 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 -- 172.21.15.110:0/1408989803 learned_addr learned my addr 172.21.15.110:0/1408989803 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc96659c640 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac20ae50 0x5618ac218a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 -- 172.21.15.110:0/1408989803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20b840 msgr2=0x5618ac218f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20b840 0x5618ac218f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 -- 172.21.15.110:0/1408989803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac20ae50 msgr2=0x5618ac218a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac20ae50 0x5618ac218a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 -- 172.21.15.110:0/1408989803 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618ac2195e0 con 0x5618ac2109b0 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc965d9b640 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20b840 0x5618ac218f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc96659c640 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac20ae50 0x5618ac218a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc966d9d640 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac2109b0 0x5618ac2852e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc960098290 tx=0x7fc9600056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd58ea39dbb93d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc94dffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc94dffb640 1 -- 172.21.15.110:0/1408989803 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9600b3020 con 0x5618ac2109b0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc94dffb640 1 -- 172.21.15.110:0/1408989803 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc9600c7940 con 0x5618ac2109b0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.660+0000 7fc96f633740 1 -- 172.21.15.110:0/1408989803 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc960084e50 con 0x5618ac2109b0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc94dffb640 1 -- 172.21.15.110:0/1408989803 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc96001f030 con 0x5618ac2109b0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96f633740 10 client.25712 fetch_fsmap learned FSMap version 32 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96f633740 1 -- 172.21.15.110:0/1408989803 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618ac286980 con 0x5618ac2109b0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc94dffb640 1 -- 172.21.15.110:0/1408989803 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc960084e50 con 0x5618ac2109b0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc94dffb640 1 -- 172.21.15.110:0/1408989803 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc96000f040 con 0x5618ac2109b0 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96f633740 10 client.25712 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96f633740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96f633740 20 client.25712 populate_metadata read hostname 'smithi110' 2024-04-25T00:13:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96f633740 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618ac220df0 0x5618ac289390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96f633740 4 client.25712 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:13:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96f633740 1 -- 172.21.15.110:0/1408989803 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618ac176540 con 0x5618ac220df0 2024-04-25T00:13:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.661+0000 7fc96659c640 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618ac220df0 0x5618ac289390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.662+0000 7fc94dffb640 10 client.25712 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.662+0000 7fc96659c640 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618ac220df0 0x5618ac289390 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f9198dc0171b3d server_cookie=a6f4b66cfc193af4 in_seq=0 out_seq=0 2024-04-25T00:13:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.676+0000 7fc94dffb640 1 -- 172.21.15.110:0/1408989803 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618ac176540 con 0x5618ac220df0 2024-04-25T00:13:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.676+0000 7fc94dffb640 10 client.25712 handle_command_reply: tid=0 2024-04-25T00:13:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.676+0000 7fc96f633740 1 client.25712 shutdown 2024-04-25T00:13:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc96f633740 1 -- 172.21.15.110:0/1408989803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac2109b0 msgr2=0x5618ac2852e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc96f633740 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac2109b0 0x5618ac2852e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc960098290 tx=0x7fc9600056c0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc96f633740 1 -- 172.21.15.110:0/1408989803 shutdown_connections 2024-04-25T00:13:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc96f633740 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618ac220df0 0x5618ac289390 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc96f633740 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618ac2109b0 0x5618ac2852e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc94dffb640 0 client.25712 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc96f633740 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618ac20b840 0x5618ac218f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc96f633740 1 --2- 172.21.15.110:0/1408989803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618ac20ae50 0x5618ac218a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.677+0000 7fc96f633740 1 -- 172.21.15.110:0/1408989803 >> 172.21.15.110:0/1408989803 conn(0x5618ac2064c0 msgr2=0x5618ac209240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.678+0000 7fc96f633740 1 -- 172.21.15.110:0/1408989803 shutdown_connections 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.678+0000 7fc96f633740 1 -- 172.21.15.110:0/1408989803 wait complete. 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.678+0000 7fc96f633740 20 client.25712 trim_cache size 0 max 16384 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:13:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:13:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:13:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:13:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9500a2d30 msgr2=0x7fc9500a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9500a2d30 0x7fc9500a51f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc96819b0d0 tx=0x7fc95c015530 comp rx=0 tx=0).stop 2024-04-25T00:13:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968101330 msgr2=0x7fc9681a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968101330 0x7fc9681a1170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc96000e940 tx=0x7fc96000ed40 comp rx=0 tx=0).stop 2024-04-25T00:13:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 shutdown_connections 2024-04-25T00:13:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9500a2d30 0x7fc9500a51f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc968101330 0x7fc9681a1170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc968100990 0x7fc96819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 --2- 172.21.15.110:0/1533677209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc968100030 0x7fc968199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 >> 172.21.15.110:0/1533677209 conn(0x7fc9680fbc10 msgr2=0x7fc9680fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 shutdown_connections 2024-04-25T00:13:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:07.679+0000 7fc96c922640 1 -- 172.21.15.110:0/1533677209 wait complete. 2024-04-25T00:13:08.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:07 smithi110 ceph-mon[29187]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-04-25T00:13:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:07 smithi173 ceph-mon[32402]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-04-25T00:13:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:07 smithi188 ceph-mon[33285]: pgmap v683: 97 pgs: 97 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-04-25T00:13:10.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:09 smithi110 ceph-mon[29187]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-04-25T00:13:10.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:09 smithi188 ceph-mon[33285]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-04-25T00:13:10.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:09 smithi173 ceph-mon[32402]: pgmap v684: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-04-25T00:13:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 605.0 s, 863.8 tps, lat 115.731 ms stddev 18.530 2024-04-25T00:13:12.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:11 smithi110 ceph-mon[29187]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 881 op/s 2024-04-25T00:13:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:11 smithi173 ceph-mon[32402]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 881 op/s 2024-04-25T00:13:12.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:11 smithi188 ceph-mon[33285]: pgmap v685: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 881 op/s 2024-04-25T00:13:14.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:13 smithi110 ceph-mon[29187]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-04-25T00:13:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:13 smithi173 ceph-mon[32402]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-04-25T00:13:14.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:13 smithi188 ceph-mon[33285]: pgmap v686: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-04-25T00:13:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:15 smithi173 ceph-mon[32402]: pgmap v687: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-04-25T00:13:16.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:15 smithi188 ceph-mon[33285]: pgmap v687: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-04-25T00:13:16.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:15 smithi110 ceph-mon[29187]: pgmap v687: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-04-25T00:13:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 610.0 s, 839.0 tps, lat 118.933 ms stddev 22.112 2024-04-25T00:13:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:17 smithi173 ceph-mon[32402]: pgmap v688: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 835 op/s 2024-04-25T00:13:18.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:17 smithi188 ceph-mon[33285]: pgmap v688: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 835 op/s 2024-04-25T00:13:18.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:17 smithi110 ceph-mon[29187]: pgmap v688: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 835 op/s 2024-04-25T00:13:20.088 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:19 smithi110 ceph-mon[29187]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-04-25T00:13:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:19 smithi173 ceph-mon[32402]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-04-25T00:13:20.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:19 smithi188 ceph-mon[33285]: pgmap v689: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-04-25T00:13:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.808+0000 7f8b9b08c640 1 -- 172.21.15.110:0/728520866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b940ffa50 msgr2=0x7f8b940ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.808+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/728520866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b940ffa50 0x7f8b940ffe70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009900 tx=0x7f8b8802a400 comp rx=0 tx=0).stop 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.809+0000 7f8b9b08c640 1 -- 172.21.15.110:0/728520866 shutdown_connections 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.809+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/728520866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b94101170 0x7f8b941019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.809+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/728520866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b941007b0 0x7f8b94100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.809+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/728520866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b940ffa50 0x7f8b940ffe70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.809+0000 7f8b9b08c640 1 -- 172.21.15.110:0/728520866 >> 172.21.15.110:0/728520866 conn(0x7f8b940fb3a0 msgr2=0x7f8b940fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.809+0000 7f8b9b08c640 1 -- 172.21.15.110:0/728520866 shutdown_connections 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.809+0000 7f8b9b08c640 1 -- 172.21.15.110:0/728520866 wait complete. 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b9b08c640 1 Processor -- start 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b9b08c640 1 -- start start 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b9b08c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b940ffa50 0x7f8b94105210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b9b08c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b941007b0 0x7f8b94105750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b9b08c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b94101170 0x7f8b9410c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b98e28640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b940ffa50 0x7f8b94105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b98e28640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b940ffa50 0x7f8b94105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:45488/0 (socket says 172.21.15.110:45488) 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b98e28640 1 -- 172.21.15.110:0/217454458 learned_addr learned my addr 172.21.15.110:0/217454458 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b99629640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b94101170 0x7f8b9410c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b93fff640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b941007b0 0x7f8b94105750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9410f4b0 con 0x7f8b94101170 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9410f330 con 0x7f8b941007b0 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.810+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9410f630 con 0x7f8b940ffa50 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b98e28640 1 -- 172.21.15.110:0/217454458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b941007b0 msgr2=0x7f8b94105750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b98e28640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b941007b0 0x7f8b94105750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b98e28640 1 -- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b94101170 msgr2=0x7f8b9410c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b98e28640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b94101170 0x7f8b9410c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b98e28640 1 -- 172.21.15.110:0/217454458 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b941ae3b0 con 0x7f8b940ffa50 2024-04-25T00:13:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b93fff640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b941007b0 0x7f8b94105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b98e28640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b940ffa50 0x7f8b94105210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005940 tx=0x7f8b880042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b91ffb640 1 -- 172.21.15.110:0/217454458 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8803f030 con 0x7f8b940ffa50 2024-04-25T00:13:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b91ffb640 1 -- 172.21.15.110:0/217454458 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b880318f0 con 0x7f8b940ffa50 2024-04-25T00:13:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b941ae640 con 0x7f8b940ffa50 2024-04-25T00:13:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.811+0000 7f8b91ffb640 1 -- 172.21.15.110:0/217454458 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88035070 con 0x7f8b940ffa50 2024-04-25T00:13:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.812+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b941aeac0 con 0x7f8b940ffa50 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.814+0000 7f8b91ffb640 1 -- 172.21.15.110:0/217454458 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8b88042030 con 0x7f8b940ffa50 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.815+0000 7f8b9dd98740 1 Processor -- start 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.815+0000 7f8b91ffb640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8b840a2cb0 0x7f8b840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.815+0000 7f8b99629640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8b840a2cb0 0x7f8b840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.815+0000 7f8b91ffb640 1 -- 172.21.15.110:0/217454458 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8b880ef3f0 con 0x7f8b940ffa50 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.815+0000 7f8b9dd98740 1 -- start start 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b561e50 0x56320b562270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b567340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b569e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56320b4c5cb0 con 0x56320b561e50 2024-04-25T00:13:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56320b46e5b0 con 0x56320b562840 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56320b4c1c70 con 0x56320b5679b0 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b99629640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8b840a2cb0 0x7f8b840a5170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005fd0 tx=0x7f8b8c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b98e28640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b561e50 0x56320b562270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b567340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b567340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:56564/0 (socket says 172.21.15.110:56564) 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 -- 172.21.15.110:0/208602952 learned_addr learned my addr 172.21.15.110:0/208602952 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b99629640 1 --2- 172.21.15.110:0/208602952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b569e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 -- 172.21.15.110:0/208602952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 msgr2=0x56320b569e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 --2- 172.21.15.110:0/208602952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b569e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 -- 172.21.15.110:0/208602952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b561e50 msgr2=0x56320b562270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 --2- 172.21.15.110:0/208602952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b561e50 0x56320b562270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 -- 172.21.15.110:0/208602952 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320b1da210 con 0x56320b562840 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.816+0000 7f8b93fff640 1 --2- 172.21.15.110:0/208602952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b567340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b800114f0 tx=0x7f8b800119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11631232b8f5fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.817+0000 7f8b7effd640 1 -- 172.21.15.110:0/208602952 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80005020 con 0x56320b562840 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.817+0000 7f8b7effd640 1 -- 172.21.15.110:0/208602952 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b800047a0 con 0x56320b562840 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.817+0000 7f8b7effd640 1 -- 172.21.15.110:0/208602952 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000f820 con 0x56320b562840 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 -- 172.21.15.110:0/208602952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 msgr2=0x56320b567340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/208602952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b567340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b800114f0 tx=0x7f8b800119b0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 -- 172.21.15.110:0/208602952 shutdown_connections 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/208602952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b569e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/208602952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b567340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/208602952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b561e50 0x56320b562270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 -- 172.21.15.110:0/208602952 >> 172.21.15.110:0/208602952 conn(0x56320b55d4c0 msgr2=0x56320b55d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 -- 172.21.15.110:0/208602952 shutdown_connections 2024-04-25T00:13:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 -- 172.21.15.110:0/208602952 wait complete. 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 -- start start 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.818+0000 7f8b9dd98740 1 Processor -- start 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 0x56320b56c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b56c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b573800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56320b56a9b0 con 0x56320b5679b0 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56320b481b50 con 0x56320b562840 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56320b56ab30 con 0x56320b561e50 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b9dd98740 10 client.0 _ll_register_callbacks cb 0x56320b4bcca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b98e28640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 0x56320b56c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b56c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b56c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:56580/0 (socket says 172.21.15.110:56580) 2024-04-25T00:13:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b93fff640 1 -- 172.21.15.110:0/4201223519 learned_addr learned my addr 172.21.15.110:0/4201223519 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b99629640 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b573800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b93fff640 1 -- 172.21.15.110:0/4201223519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 msgr2=0x56320b56c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b93fff640 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 0x56320b56c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b93fff640 1 -- 172.21.15.110:0/4201223519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 msgr2=0x56320b573800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b93fff640 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b573800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b93fff640 1 -- 172.21.15.110:0/4201223519 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320b2a6620 con 0x56320b562840 2024-04-25T00:13:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.819+0000 7f8b99629640 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b573800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.820+0000 7f8b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.820+0000 7f8b93fff640 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b56c7c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009380 tx=0x7f8b80005210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8408860e8715928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.820+0000 7f8b73fff640 1 -- 172.21.15.110:0/4201223519 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80028070 con 0x56320b562840 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.820+0000 7f8b9dd98740 1 -- 172.21.15.110:0/4201223519 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b800047a0 con 0x56320b562840 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.820+0000 7f8b73fff640 1 -- 172.21.15.110:0/4201223519 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8b8000f620 con 0x56320b562840 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b73fff640 1 -- 172.21.15.110:0/4201223519 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8b80032050 con 0x56320b562840 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b9dd98740 10 client.25724 fetch_fsmap learned FSMap version 32 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b9dd98740 1 -- 172.21.15.110:0/4201223519 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56320b576b70 con 0x56320b562840 2024-04-25T00:13:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b73fff640 1 -- 172.21.15.110:0/4201223519 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b800047a0 con 0x56320b562840 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b73fff640 1 -- 172.21.15.110:0/4201223519 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f8b8002d6a0 con 0x56320b562840 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b9dd98740 10 client.25724 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b9dd98740 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b9dd98740 20 client.25724 populate_metadata read hostname 'smithi110' 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56320b5783e0 0x56320b57a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b9dd98740 4 client.25724 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.821+0000 7f8b9dd98740 1 -- 172.21.15.110:0/4201223519 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56320b4d31f0 con 0x56320b5783e0 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.822+0000 7f8b98e28640 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56320b5783e0 0x56320b57a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.822+0000 7f8b73fff640 10 client.25724 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.823+0000 7f8b98e28640 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56320b5783e0 0x56320b57a820 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff52a24596f09245 server_cookie=38f0aded9e93ce81 in_seq=0 out_seq=0 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.823+0000 7f8b73fff640 1 -- 172.21.15.110:0/4201223519 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56320b4d31f0 con 0x56320b5783e0 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.823+0000 7f8b73fff640 10 client.25724 handle_command_reply: tid=0 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.823+0000 7f8b9dd98740 1 client.25724 shutdown 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.823+0000 7f8b9dd98740 1 -- 172.21.15.110:0/4201223519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 msgr2=0x56320b56c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.823+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b56c7c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009380 tx=0x7f8b80005210 comp rx=0 tx=0).stop 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 1 -- 172.21.15.110:0/4201223519 shutdown_connections 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56320b5783e0 0x56320b57a820 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b573800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b56c7c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/4201223519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 0x56320b56c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 1 -- 172.21.15.110:0/4201223519 >> 172.21.15.110:0/4201223519 conn(0x56320b55d4c0 msgr2=0x56320b55c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b73fff640 0 client.25724 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 1 -- 172.21.15.110:0/4201223519 shutdown_connections 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 1 -- 172.21.15.110:0/4201223519 wait complete. 2024-04-25T00:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.824+0000 7f8b9dd98740 20 client.25724 trim_cache size 0 max 16384 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b9dd98740 1 Processor -- start 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b9dd98740 1 -- start start 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b561e50 0x56320b560c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b562840 0x56320b5737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b57f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941ba140 con 0x56320b562840 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941b7a10 con 0x56320b561e50 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941b5860 con 0x56320b5679b0 2024-04-25T00:13:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b562840 0x56320b5737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b562840 0x56320b5737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53360/0 (socket says 172.21.15.110:53360) 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b93fff640 1 -- 172.21.15.110:0/1427915599 learned_addr learned my addr 172.21.15.110:0/1427915599 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b99629640 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b57f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.850+0000 7f8b98e28640 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b561e50 0x56320b560c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b99629640 1 -- 172.21.15.110:0/1427915599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b561e50 msgr2=0x56320b560c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b99629640 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b561e50 0x56320b560c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b99629640 1 -- 172.21.15.110:0/1427915599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b562840 msgr2=0x56320b5737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b99629640 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b562840 0x56320b5737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b99629640 1 -- 172.21.15.110:0/1427915599 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320b563060 con 0x56320b5679b0 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b98e28640 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b561e50 0x56320b560c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b93fff640 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b562840 0x56320b5737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b99629640 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b57f0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c011ca0 tx=0x7f8b8c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7edbb36d1854b086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b7e7fc640 1 -- 172.21.15.110:0/1427915599 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c003a30 con 0x56320b5679b0 2024-04-25T00:13:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b7e7fc640 1 -- 172.21.15.110:0/1427915599 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b8c003bd0 con 0x56320b5679b0 2024-04-25T00:13:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.851+0000 7f8b7e7fc640 1 -- 172.21.15.110:0/1427915599 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c01f850 con 0x56320b5679b0 2024-04-25T00:13:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.852+0000 7f8b9dd98740 1 -- 172.21.15.110:0/1427915599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 msgr2=0x56320b57f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.852+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b57f0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c011ca0 tx=0x7f8b8c014000 comp rx=0 tx=0).stop 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.852+0000 7f8b9dd98740 1 -- 172.21.15.110:0/1427915599 shutdown_connections 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.852+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b5679b0 0x56320b57f0e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.852+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b562840 0x56320b5737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.852+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/1427915599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b561e50 0x56320b560c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.853+0000 7f8b9dd98740 1 -- 172.21.15.110:0/1427915599 >> 172.21.15.110:0/1427915599 conn(0x56320b55d4c0 msgr2=0x56320b560520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.853+0000 7f8b9dd98740 1 -- 172.21.15.110:0/1427915599 shutdown_connections 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.853+0000 7f8b9dd98740 1 -- 172.21.15.110:0/1427915599 wait complete. 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.853+0000 7f8b9dd98740 1 -- start start 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.853+0000 7f8b9dd98740 1 Processor -- start 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 0x56320b570dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b571310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b9dd98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b572160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941b5ed0 con 0x56320b5679b0 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941b7340 con 0x56320b562840 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b9dd98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941b65a0 con 0x56320b561e50 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b9dd98740 10 client.0 _ll_register_callbacks cb 0x7f8b88009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b99629640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b572160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b98e28640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 0x56320b570dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b571310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b99629640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b572160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53372/0 (socket says 172.21.15.110:53372) 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b99629640 1 -- 172.21.15.110:0/916631229 learned_addr learned my addr 172.21.15.110:0/916631229 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b99629640 1 -- 172.21.15.110:0/916631229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 msgr2=0x56320b570dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.854+0000 7f8b99629640 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 0x56320b570dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b99629640 1 -- 172.21.15.110:0/916631229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 msgr2=0x56320b571310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b99629640 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b571310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b99629640 1 -- 172.21.15.110:0/916631229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320b572750 con 0x56320b5679b0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b99629640 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b572160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c003fb0 tx=0x7f8b8c004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d869df10bd02a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b73fff640 1 -- 172.21.15.110:0/916631229 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c004540 con 0x56320b5679b0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b9dd98740 1 -- 172.21.15.110:0/916631229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8c003bd0 con 0x56320b5679b0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b73fff640 1 -- 172.21.15.110:0/916631229 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f8b8c025040 con 0x56320b5679b0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b73fff640 1 -- 172.21.15.110:0/916631229 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8b8c01f790 con 0x56320b5679b0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b9dd98740 10 client.16146 fetch_fsmap learned FSMap version 32 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b9dd98740 1 -- 172.21.15.110:0/916631229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56320b573a60 con 0x56320b5679b0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.855+0000 7f8b73fff640 1 -- 172.21.15.110:0/916631229 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b8c003bd0 con 0x56320b5679b0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b73fff640 1 -- 172.21.15.110:0/916631229 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f8b8c03dae0 con 0x56320b5679b0 2024-04-25T00:13:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b9dd98740 10 client.16146 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:13:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b9dd98740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:13:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b9dd98740 20 client.16146 populate_metadata read hostname 'smithi110' 2024-04-25T00:13:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56320b5df000 0x56320b5e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b9dd98740 4 client.16146 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:13:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b9dd98740 1 -- 172.21.15.110:0/916631229 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56320b4cd540 con 0x56320b5df000 2024-04-25T00:13:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b98e28640 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56320b5df000 0x56320b5e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b73fff640 10 client.16146 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.856+0000 7f8b98e28640 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56320b5df000 0x56320b5e1420 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6fd5bf06b735eb server_cookie=9ba32684f9e19ca2 in_seq=0 out_seq=0 2024-04-25T00:13:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b73fff640 1 -- 172.21.15.110:0/916631229 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56320b4cd540 con 0x56320b5df000 2024-04-25T00:13:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b73fff640 10 client.16146 handle_command_reply: tid=0 2024-04-25T00:13:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 client.16146 shutdown 2024-04-25T00:13:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 -- 172.21.15.110:0/916631229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 msgr2=0x56320b572160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b572160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c003fb0 tx=0x7f8b8c004050 comp rx=0 tx=0).stop 2024-04-25T00:13:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 -- 172.21.15.110:0/916631229 shutdown_connections 2024-04-25T00:13:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56320b5df000 0x56320b5e1420 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56320b5679b0 0x56320b572160 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b73fff640 0 client.16146 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56320b562840 0x56320b571310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 --2- 172.21.15.110:0/916631229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56320b561e50 0x56320b570dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.869+0000 7f8b9dd98740 1 -- 172.21.15.110:0/916631229 >> 172.21.15.110:0/916631229 conn(0x56320b55d4c0 msgr2=0x56320b55bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.870+0000 7f8b9dd98740 1 -- 172.21.15.110:0/916631229 shutdown_connections 2024-04-25T00:13:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.870+0000 7f8b9dd98740 1 -- 172.21.15.110:0/916631229 wait complete. 2024-04-25T00:13:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.870+0000 7f8b9dd98740 20 client.16146 trim_cache size 0 max 16384 2024-04-25T00:13:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:13:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:13:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:13:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:13:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8b840a2cb0 msgr2=0x7f8b840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8b840a2cb0 0x7f8b840a5170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005fd0 tx=0x7f8b8c009500 comp rx=0 tx=0).stop 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b940ffa50 msgr2=0x7f8b94105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b940ffa50 0x7f8b94105210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005940 tx=0x7f8b880042f0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 shutdown_connections 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f8b840a2cb0 0x7f8b840a5170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b94101170 0x7f8b9410c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b941007b0 0x7f8b94105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 --2- 172.21.15.110:0/217454458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b940ffa50 0x7f8b94105210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 >> 172.21.15.110:0/217454458 conn(0x7f8b940fb3a0 msgr2=0x7f8b940fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 shutdown_connections 2024-04-25T00:13:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:20.871+0000 7f8b9b08c640 1 -- 172.21.15.110:0/217454458 wait complete. 2024-04-25T00:13:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 615.0 s, 833.6 tps, lat 120.238 ms stddev 21.376 2024-04-25T00:13:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:21 smithi173 ceph-mon[32402]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 819 op/s 2024-04-25T00:13:22.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:21 smithi188 ceph-mon[33285]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 819 op/s 2024-04-25T00:13:22.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:21 smithi110 ceph-mon[29187]: pgmap v690: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 819 op/s 2024-04-25T00:13:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:23 smithi173 ceph-mon[32402]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-04-25T00:13:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:23 smithi188 ceph-mon[33285]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-04-25T00:13:24.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:23 smithi110 ceph-mon[29187]: pgmap v691: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2024-04-25T00:13:25.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:24 smithi173 ceph-mon[32402]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.3 MiB/s wr, 811 op/s 2024-04-25T00:13:25.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:24 smithi188 ceph-mon[33285]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.3 MiB/s wr, 811 op/s 2024-04-25T00:13:25.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:24 smithi110 ceph-mon[29187]: pgmap v692: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.3 MiB/s wr, 811 op/s 2024-04-25T00:13:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 620.0 s, 810.6 tps, lat 122.982 ms stddev 18.597 2024-04-25T00:13:28.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:27 smithi110 ceph-mon[29187]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 746 op/s 2024-04-25T00:13:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:27 smithi173 ceph-mon[32402]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 746 op/s 2024-04-25T00:13:28.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:27 smithi188 ceph-mon[33285]: pgmap v693: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 746 op/s 2024-04-25T00:13:30.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:29 smithi110 ceph-mon[29187]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-04-25T00:13:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:29 smithi173 ceph-mon[32402]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-04-25T00:13:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:29 smithi188 ceph-mon[33285]: pgmap v694: 97 pgs: 97 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-04-25T00:13:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 625.0 s, 853.2 tps, lat 117.073 ms stddev 16.257 2024-04-25T00:13:32.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:31 smithi110 ceph-mon[29187]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.6 MiB/s wr, 861 op/s 2024-04-25T00:13:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:31 smithi173 ceph-mon[32402]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.6 MiB/s wr, 861 op/s 2024-04-25T00:13:32.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:31 smithi188 ceph-mon[33285]: pgmap v695: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.6 MiB/s wr, 861 op/s 2024-04-25T00:13:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 -- 172.21.15.110:0/2211143379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 msgr2=0x7f5384101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 --2- 172.21.15.110:0/2211143379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 0x7f5384101be0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5374009900 tx=0x7f537402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 -- 172.21.15.110:0/2211143379 shutdown_connections 2024-04-25T00:13:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 --2- 172.21.15.110:0/2211143379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5384103370 0x7f5384109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 --2- 172.21.15.110:0/2211143379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53841029b0 0x7f5384102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 --2- 172.21.15.110:0/2211143379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 0x7f5384101be0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 -- 172.21.15.110:0/2211143379 >> 172.21.15.110:0/2211143379 conn(0x7f53840fcd30 msgr2=0x7f53840ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 -- 172.21.15.110:0/2211143379 shutdown_connections 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 -- 172.21.15.110:0/2211143379 wait complete. 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 Processor -- start 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.998+0000 7f538a309640 1 -- start start 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f538a309640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 0x7f538419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f538a309640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53841029b0 0x7f538419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f538a309640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5384103370 0x7f53841a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f538a309640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f538410d310 con 0x7f53841029b0 2024-04-25T00:13:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f538a309640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f538410d190 con 0x7f53841017c0 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f538a309640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f538410d490 con 0x7f5384103370 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f53837fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53841029b0 0x7f538419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f5383fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 0x7f538419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f53837fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53841029b0 0x7f538419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38196/0 (socket says 172.21.15.110:38196) 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f5383fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 0x7f538419dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:56710/0 (socket says 172.21.15.110:56710) 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f53837fe640 1 -- 172.21.15.110:0/3914150598 learned_addr learned my addr 172.21.15.110:0/3914150598 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f53888a6640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5384103370 0x7f53841a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f5383fff640 1 -- 172.21.15.110:0/3914150598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5384103370 msgr2=0x7f53841a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f5383fff640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5384103370 0x7f53841a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f5383fff640 1 -- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53841029b0 msgr2=0x7f538419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f5383fff640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53841029b0 0x7f538419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f5383fff640 1 -- 172.21.15.110:0/3914150598 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53841a5b00 con 0x7f53841017c0 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:33.999+0000 7f53888a6640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5384103370 0x7f53841a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.000+0000 7f5383fff640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 0x7f538419dee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5374009230 tx=0x7f5374003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.000+0000 7f53817fa640 1 -- 172.21.15.110:0/3914150598 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537403f070 con 0x7f53841017c0 2024-04-25T00:13:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.000+0000 7f53817fa640 1 -- 172.21.15.110:0/3914150598 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5374031c40 con 0x7f53841017c0 2024-04-25T00:13:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.000+0000 7f53817fa640 1 -- 172.21.15.110:0/3914150598 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374035030 con 0x7f53841017c0 2024-04-25T00:13:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.000+0000 7f538a309640 1 -- 172.21.15.110:0/3914150598 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53841a5d90 con 0x7f53841017c0 2024-04-25T00:13:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.000+0000 7f538a309640 1 -- 172.21.15.110:0/3914150598 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53841a61a0 con 0x7f53841017c0 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.003+0000 7f53817fa640 1 -- 172.21.15.110:0/3914150598 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5374042070 con 0x7f53841017c0 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.003+0000 7f538d010740 1 Processor -- start 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.003+0000 7f538d010740 1 -- start start 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f538d010740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf899270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f538d010740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf899840 0x556cdf89e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f538d010740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf89e9b0 0x556cdf8a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f538d010740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556cdf7fccb0 con 0x556cdf898e50 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f538d010740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556cdf7a55b0 con 0x556cdf89e9b0 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f538d010740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556cdf7f8c70 con 0x556cdf899840 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf89e9b0 0x556cdf8a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf89e9b0 0x556cdf8a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:56722/0 (socket says 172.21.15.110:56722) 2024-04-25T00:13:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 -- 172.21.15.110:0/4239304502 learned_addr learned my addr 172.21.15.110:0/4239304502 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53888a6640 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf899270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f5383fff640 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf899840 0x556cdf89e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53817fa640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f536c0a2c30 0x7f536c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 -- 172.21.15.110:0/4239304502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf899840 msgr2=0x556cdf89e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf899840 0x556cdf89e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 -- 172.21.15.110:0/4239304502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 msgr2=0x556cdf899270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf899270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 -- 172.21.15.110:0/4239304502 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdf511210 con 0x556cdf89e9b0 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53817fa640 1 -- 172.21.15.110:0/3914150598 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f53740ef720 con 0x7f53841017c0 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.004+0000 7f53837fe640 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf89e9b0 0x556cdf8a0e40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f537800ad50 tx=0x7f537800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1dab553d4063080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.005+0000 7f536a7fc640 1 -- 172.21.15.110:0/4239304502 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537801d070 con 0x556cdf89e9b0 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.005+0000 7f53888a6640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f536c0a2c30 0x7f536c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.005+0000 7f536a7fc640 1 -- 172.21.15.110:0/4239304502 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f537800ae40 con 0x556cdf89e9b0 2024-04-25T00:13:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.005+0000 7f536a7fc640 1 -- 172.21.15.110:0/4239304502 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378018ba0 con 0x556cdf89e9b0 2024-04-25T00:13:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.005+0000 7f538d010740 1 -- 172.21.15.110:0/4239304502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf89e9b0 msgr2=0x556cdf8a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.005+0000 7f538d010740 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf89e9b0 0x556cdf8a0e40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f537800ad50 tx=0x7f537800f790 comp rx=0 tx=0).stop 2024-04-25T00:13:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.005+0000 7f53888a6640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f536c0a2c30 0x7f536c0a50f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x556cdf89a0a0 tx=0x7f537c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 -- 172.21.15.110:0/4239304502 shutdown_connections 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf89e9b0 0x556cdf8a0e40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf899840 0x556cdf89e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 --2- 172.21.15.110:0/4239304502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf899270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 -- 172.21.15.110:0/4239304502 >> 172.21.15.110:0/4239304502 conn(0x556cdf8944c0 msgr2=0x556cdf8948c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 -- 172.21.15.110:0/4239304502 shutdown_connections 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 -- 172.21.15.110:0/4239304502 wait complete. 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 -- start start 2024-04-25T00:13:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.006+0000 7f538d010740 1 Processor -- start 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f538d010740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f538d010740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f538d010740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f538d010740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556cdf8a1960 con 0x556cdf898e50 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f538d010740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556cdf7b8b50 con 0x556cdf899840 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f538d010740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556cdf8a1ae0 con 0x556cdf89e9b0 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f538d010740 10 client.0 _ll_register_callbacks cb 0x556cdf7f3ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f5383fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f5383fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38220/0 (socket says 172.21.15.110:38220) 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f5383fff640 1 -- 172.21.15.110:0/1473828676 learned_addr learned my addr 172.21.15.110:0/1473828676 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f53888a6640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.007+0000 7f53837fe640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f5383fff640 1 -- 172.21.15.110:0/1473828676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 msgr2=0x556cdf8aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f5383fff640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8aa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f5383fff640 1 -- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 msgr2=0x556cdf8a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f5383fff640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f5383fff640 1 -- 172.21.15.110:0/1473828676 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdf5dd620 con 0x556cdf898e50 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f53837fe640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f53888a6640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f5383fff640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a3270 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5374000c00 tx=0x7f537403b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14380e070761f96a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f53637fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f53637fe640 1 -- 172.21.15.110:0/1473828676 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374049070 con 0x556cdf898e50 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f538d010740 1 -- 172.21.15.110:0/1473828676 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556cdf80a1f0 con 0x556cdf898e50 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.008+0000 7f53637fe640 1 -- 172.21.15.110:0/1473828676 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f537403b580 con 0x556cdf898e50 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f53637fe640 1 -- 172.21.15.110:0/1473828676 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53740bbba0 con 0x556cdf898e50 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f538d010740 10 client.16152 fetch_fsmap learned FSMap version 32 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f538d010740 1 -- 172.21.15.110:0/1473828676 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556cdf8adb60 con 0x556cdf898e50 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f53637fe640 1 -- 172.21.15.110:0/1473828676 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556cdf80a1f0 con 0x556cdf898e50 2024-04-25T00:13:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f53637fe640 1 -- 172.21.15.110:0/1473828676 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f53740b6bd0 con 0x556cdf898e50 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f538d010740 10 client.16152 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f538d010740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f538d010740 20 client.16152 populate_metadata read hostname 'smithi110' 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f538d010740 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556cdf8af360 0x556cdf8b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f538d010740 4 client.16152 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.009+0000 7f538d010740 1 -- 172.21.15.110:0/1473828676 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cdf804540 con 0x556cdf8af360 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.010+0000 7f53837fe640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556cdf8af360 0x556cdf8b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.010+0000 7f53637fe640 10 client.16152 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.010+0000 7f53837fe640 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556cdf8af360 0x556cdf8b17f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b059b0355f1b120c server_cookie=7c5c9204517c4a9c in_seq=0 out_seq=0 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f53637fe640 1 -- 172.21.15.110:0/1473828676 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cdf804540 con 0x556cdf8af360 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f53637fe640 10 client.16152 handle_command_reply: tid=0 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 client.16152 shutdown 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 -- 172.21.15.110:0/1473828676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 msgr2=0x556cdf8a3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a3270 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5374000c00 tx=0x7f537403b460 comp rx=0 tx=0).stop 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 -- 172.21.15.110:0/1473828676 shutdown_connections 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556cdf8af360 0x556cdf8b17f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f53637fe640 0 client.16152 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 --2- 172.21.15.110:0/1473828676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a3270 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 -- 172.21.15.110:0/1473828676 >> 172.21.15.110:0/1473828676 conn(0x556cdf8944c0 msgr2=0x556cdf8935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.011+0000 7f538d010740 1 -- 172.21.15.110:0/1473828676 shutdown_connections 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.012+0000 7f538d010740 1 -- 172.21.15.110:0/1473828676 wait complete. 2024-04-25T00:13:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.012+0000 7f538d010740 20 client.16152 trim_cache size 0 max 16384 2024-04-25T00:13:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f538d010740 1 Processor -- start 2024-04-25T00:13:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f538d010740 1 -- start start 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f538d010740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f538d010740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f538d010740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f538d010740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53841b1810 con 0x556cdf898e50 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f538d010740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53841af0e0 con 0x556cdf899840 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f538d010740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53841acf30 con 0x556cdf89e9b0 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f5383fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f53837fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f5383fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38236/0 (socket says 172.21.15.110:38236) 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f5383fff640 1 -- 172.21.15.110:0/1144723294 learned_addr learned my addr 172.21.15.110:0/1144723294 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f53888a6640 1 --2- 172.21.15.110:0/1144723294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f53837fe640 1 -- 172.21.15.110:0/1144723294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 msgr2=0x556cdf8a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f53837fe640 1 --2- 172.21.15.110:0/1144723294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8a33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f53837fe640 1 -- 172.21.15.110:0/1144723294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 msgr2=0x556cdf8a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f53837fe640 1 --2- 172.21.15.110:0/1144723294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.035+0000 7f53837fe640 1 -- 172.21.15.110:0/1144723294 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdf894060 con 0x556cdf899840 2024-04-25T00:13:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.036+0000 7f53837fe640 1 --2- 172.21.15.110:0/1144723294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8aa7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f537800fc90 tx=0x7f5378005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3818b417db4d13c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:33 smithi110 ceph-mon[29187]: pgmap v696: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.036+0000 7f5369ffb640 1 -- 172.21.15.110:0/1144723294 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537801d040 con 0x556cdf899840 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.036+0000 7f5369ffb640 1 -- 172.21.15.110:0/1144723294 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f537801e040 con 0x556cdf899840 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.036+0000 7f5369ffb640 1 -- 172.21.15.110:0/1144723294 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53780193c0 con 0x556cdf899840 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.037+0000 7f538d010740 1 -- 172.21.15.110:0/1144723294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 msgr2=0x556cdf8aa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.037+0000 7f538d010740 1 --2- 172.21.15.110:0/1144723294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8aa7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f537800fc90 tx=0x7f5378005f60 comp rx=0 tx=0).stop 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.037+0000 7f538d010740 1 -- 172.21.15.110:0/1144723294 shutdown_connections 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.037+0000 7f538d010740 1 --2- 172.21.15.110:0/1144723294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.037+0000 7f538d010740 1 --2- 172.21.15.110:0/1144723294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf899840 0x556cdf8aa7b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.037+0000 7f538d010740 1 --2- 172.21.15.110:0/1144723294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf898e50 0x556cdf8a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.038+0000 7f538d010740 1 -- 172.21.15.110:0/1144723294 >> 172.21.15.110:0/1144723294 conn(0x556cdf8944c0 msgr2=0x556cdf8948a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.038+0000 7f538d010740 1 -- 172.21.15.110:0/1144723294 shutdown_connections 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.038+0000 7f538d010740 1 -- 172.21.15.110:0/1144723294 wait complete. 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f538d010740 1 -- start start 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f538d010740 1 Processor -- start 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f538d010740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf898e50 0x556cdf8aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f538d010740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf899840 0x556cdf8a7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f538d010740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8a7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f538d010740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f53841ad5a0 con 0x556cdf899840 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f538d010740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53841aea10 con 0x556cdf898e50 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f538d010740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53841adc70 con 0x556cdf89e9b0 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf898e50 0x556cdf8aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.039+0000 7f53837fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf899840 0x556cdf8a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf898e50 0x556cdf8aa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:56748/0 (socket says 172.21.15.110:56748) 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 -- 172.21.15.110:0/1045818241 learned_addr learned my addr 172.21.15.110:0/1045818241 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f53888a6640 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f538d010740 10 client.0 _ll_register_callbacks cb 0x7f53740bbf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 -- 172.21.15.110:0/1045818241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 msgr2=0x556cdf8a7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8a7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 -- 172.21.15.110:0/1045818241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf899840 msgr2=0x556cdf8a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf899840 0x556cdf8a7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 -- 172.21.15.110:0/1045818241 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdf9139e0 con 0x556cdf898e50 2024-04-25T00:13:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f5383fff640 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf898e50 0x556cdf8aa7b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53740ed5c0 tx=0x7f53740b6f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9199fbd9e2a3893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f53637fe640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f53637fe640 1 -- 172.21.15.110:0/1045818241 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374049040 con 0x556cdf898e50 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f53637fe640 1 -- 172.21.15.110:0/1045818241 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f53740c4790 con 0x556cdf898e50 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.040+0000 7f538d010740 1 -- 172.21.15.110:0/1045818241 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537801e040 con 0x556cdf898e50 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f53637fe640 1 -- 172.21.15.110:0/1045818241 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53740ca020 con 0x556cdf898e50 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f538d010740 10 client.25748 fetch_fsmap learned FSMap version 32 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f538d010740 1 -- 172.21.15.110:0/1045818241 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556cdf9139e0 con 0x556cdf898e50 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f53637fe640 1 -- 172.21.15.110:0/1045818241 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f537801e040 con 0x556cdf898e50 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f53637fe640 1 -- 172.21.15.110:0/1045818241 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5374031790 con 0x556cdf898e50 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f538d010740 10 client.25748 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:13:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f538d010740 10 client.25748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:13:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f538d010740 20 client.25748 populate_metadata read hostname 'smithi110' 2024-04-25T00:13:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f538d010740 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556cdf915320 0x556cdf917740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f538d010740 4 client.25748 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:13:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.041+0000 7f538d010740 1 -- 172.21.15.110:0/1045818241 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cdf7828e0 con 0x556cdf915320 2024-04-25T00:13:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.042+0000 7f53837fe640 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556cdf915320 0x556cdf917740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.042+0000 7f53637fe640 10 client.25748 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.042+0000 7f53837fe640 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556cdf915320 0x556cdf917740 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8865f2d59b3e60 server_cookie=bd8d5db721d4cd2b in_seq=0 out_seq=0 2024-04-25T00:13:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.058+0000 7f53637fe640 1 -- 172.21.15.110:0/1045818241 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cdf7828e0 con 0x556cdf915320 2024-04-25T00:13:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.058+0000 7f53637fe640 10 client.25748 handle_command_reply: tid=0 2024-04-25T00:13:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.058+0000 7f538d010740 1 client.25748 shutdown 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.058+0000 7f538d010740 1 -- 172.21.15.110:0/1045818241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf898e50 msgr2=0x556cdf8aa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.058+0000 7f538d010740 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf898e50 0x556cdf8aa7b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53740ed5c0 tx=0x7f53740b6f90 comp rx=0 tx=0).stop 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.058+0000 7f538d010740 1 -- 172.21.15.110:0/1045818241 shutdown_connections 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.058+0000 7f538d010740 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x556cdf915320 0x556cdf917740 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.059+0000 7f538d010740 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556cdf89e9b0 0x556cdf8a7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.059+0000 7f538d010740 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdf899840 0x556cdf8a7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.059+0000 7f538d010740 1 --2- 172.21.15.110:0/1045818241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556cdf898e50 0x556cdf8aa7b0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53740ed5c0 tx=0x7f53740b6f90 comp rx=0 tx=0).stop 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.059+0000 7f538d010740 1 -- 172.21.15.110:0/1045818241 >> 172.21.15.110:0/1045818241 conn(0x556cdf8944c0 msgr2=0x556cdf8a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.059+0000 7f53637fe640 0 client.25748 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.060+0000 7f538d010740 1 -- 172.21.15.110:0/1045818241 shutdown_connections 2024-04-25T00:13:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.060+0000 7f538d010740 1 -- 172.21.15.110:0/1045818241 wait complete. 2024-04-25T00:13:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.060+0000 7f538d010740 20 client.25748 trim_cache size 0 max 16384 2024-04-25T00:13:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:13:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:13:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:13:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:13:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 -- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f536c0a2c30 msgr2=0x7f536c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f536c0a2c30 0x7f536c0a50f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x556cdf89a0a0 tx=0x7f537c004ab0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 -- 172.21.15.110:0/3914150598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 msgr2=0x7f538419dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 0x7f538419dee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5374009230 tx=0x7f5374003c10 comp rx=0 tx=0).stop 2024-04-25T00:13:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 -- 172.21.15.110:0/3914150598 shutdown_connections 2024-04-25T00:13:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f536c0a2c30 0x7f536c0a50f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5384103370 0x7f53841a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f53841029b0 0x7f538419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 --2- 172.21.15.110:0/3914150598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53841017c0 0x7f538419dee0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 -- 172.21.15.110:0/3914150598 >> 172.21.15.110:0/3914150598 conn(0x7f53840fcd30 msgr2=0x7f5384103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 -- 172.21.15.110:0/3914150598 shutdown_connections 2024-04-25T00:13:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:34.062+0000 7f538a309640 1 -- 172.21.15.110:0/3914150598 wait complete. 2024-04-25T00:13:34.108 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:33 smithi173 ceph-mon[32402]: pgmap v696: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-04-25T00:13:34.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:33 smithi188 ceph-mon[33285]: pgmap v696: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-04-25T00:13:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:35 smithi173 ceph-mon[32402]: pgmap v697: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 888 op/s 2024-04-25T00:13:36.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:35 smithi188 ceph-mon[33285]: pgmap v697: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 888 op/s 2024-04-25T00:13:36.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:35 smithi110 ceph-mon[29187]: pgmap v697: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 888 op/s 2024-04-25T00:13:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 630.0 s, 859.8 tps, lat 116.639 ms stddev 16.320 2024-04-25T00:13:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:37 smithi173 ceph-mon[32402]: pgmap v698: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-04-25T00:13:38.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:37 smithi188 ceph-mon[33285]: pgmap v698: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-04-25T00:13:38.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:37 smithi110 ceph-mon[29187]: pgmap v698: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-04-25T00:13:40.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:39 smithi188 ceph-mon[33285]: pgmap v699: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-04-25T00:13:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:39 smithi173 ceph-mon[32402]: pgmap v699: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-04-25T00:13:40.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:39 smithi110 ceph-mon[29187]: pgmap v699: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-04-25T00:13:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 635.0 s, 866.2 tps, lat 115.420 ms stddev 16.569 2024-04-25T00:13:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:41 smithi173 ceph-mon[32402]: pgmap v700: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 900 op/s 2024-04-25T00:13:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:41 smithi188 ceph-mon[33285]: pgmap v700: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 900 op/s 2024-04-25T00:13:42.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:41 smithi110 ceph-mon[29187]: pgmap v700: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 900 op/s 2024-04-25T00:13:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:43 smithi173 ceph-mon[32402]: pgmap v701: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.13k op/s 2024-04-25T00:13:44.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:43 smithi188 ceph-mon[33285]: pgmap v701: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.13k op/s 2024-04-25T00:13:44.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:43 smithi110 ceph-mon[29187]: pgmap v701: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.13k op/s 2024-04-25T00:13:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:45 smithi173 ceph-mon[32402]: pgmap v702: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 955 op/s 2024-04-25T00:13:46.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:45 smithi188 ceph-mon[33285]: pgmap v702: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 955 op/s 2024-04-25T00:13:46.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:45 smithi110 ceph-mon[29187]: pgmap v702: 97 pgs: 97 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 955 op/s 2024-04-25T00:13:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 640.0 s, 852.8 tps, lat 117.371 ms stddev 14.749 2024-04-25T00:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 -- 172.21.15.110:0/1029610648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 msgr2=0x7fdaac1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 --2- 172.21.15.110:0/1029610648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 0x7fdaac1098d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009860 tx=0x7fdaa002a340 comp rx=0 tx=0).stop 2024-04-25T00:13:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 -- 172.21.15.110:0/1029610648 shutdown_connections 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 --2- 172.21.15.110:0/1029610648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 0x7fdaac1098d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 --2- 172.21.15.110:0/1029610648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac104bb0 0x7fdaac106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 --2- 172.21.15.110:0/1029610648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac102280 0x7fdaac104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 -- 172.21.15.110:0/1029610648 >> 172.21.15.110:0/1029610648 conn(0x7fdaac0fbc10 msgr2=0x7fdaac0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 -- 172.21.15.110:0/1029610648 shutdown_connections 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.193+0000 7fdab085d640 1 -- 172.21.15.110:0/1029610648 wait complete. 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.194+0000 7fdab085d640 1 Processor -- start 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.194+0000 7fdab085d640 1 -- start start 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdab085d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac102280 0x7fdaac196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdab085d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac104bb0 0x7fdaac196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdab085d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 0x7fdaac192eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdab085d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10d2f0 con 0x7fdaac1074e0 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdab085d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10d170 con 0x7fdaac102280 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdab085d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10d470 con 0x7fdaac104bb0 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaa9d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac104bb0 0x7fdaac196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaaa59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac102280 0x7fdaac196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaa9d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac104bb0 0x7fdaac196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:44082/0 (socket says 172.21.15.110:44082) 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/2614434693 learned_addr learned my addr 172.21.15.110:0/2614434693 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaaa59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac102280 0x7fdaac196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:57462/0 (socket says 172.21.15.110:57462) 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaaad9d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 0x7fdaac192eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/2614434693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac102280 msgr2=0x7fdaac196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac102280 0x7fdaac196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 msgr2=0x7fdaac192eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 0x7fdaac192eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/2614434693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac193770 con 0x7fdaac104bb0 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac102280 0x7fdaac196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.195+0000 7fdaaad9d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 0x7fdaac192eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.196+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac104bb0 0x7fdaac196880 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00e9f0 tx=0x7fda9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.196+0000 7fda9b7fe640 1 -- 172.21.15.110:0/2614434693 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00ccc0 con 0x7fdaac104bb0 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.196+0000 7fda9b7fe640 1 -- 172.21.15.110:0/2614434693 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda9c0054d0 con 0x7fdaac104bb0 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.196+0000 7fdab085d640 1 -- 172.21.15.110:0/2614434693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac1a5f90 con 0x7fdaac104bb0 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.196+0000 7fda9b7fe640 1 -- 172.21.15.110:0/2614434693 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c010430 con 0x7fdaac104bb0 2024-04-25T00:13:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.196+0000 7fdab085d640 1 -- 172.21.15.110:0/2614434693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a6340 con 0x7fdaac104bb0 2024-04-25T00:13:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.199+0000 7fdab3569740 1 Processor -- start 2024-04-25T00:13:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.199+0000 7fda9b7fe640 1 -- 172.21.15.110:0/2614434693 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fda9c005020 con 0x7fdaac104bb0 2024-04-25T00:13:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.199+0000 7fdab3569740 1 -- start start 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdab3569740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df98e50 0x56000df99270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdab3569740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df99840 0x56000df9e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdab3569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df9e9b0 0x56000dfa0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdab3569740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56000defccb0 con 0x56000df98e50 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdab3569740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56000dea55b0 con 0x56000df99840 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdab3569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56000def8c70 con 0x56000df9e9b0 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaaad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df98e50 0x56000df99270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaa9d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df9e9b0 0x56000dfa0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaa9d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df9e9b0 0x56000dfa0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:44098/0 (socket says 172.21.15.110:44098) 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaaad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df98e50 0x56000df99270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40230/0 (socket says 172.21.15.110:40230) 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/1625214755 learned_addr learned my addr 172.21.15.110:0/1625214755 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/1625214755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df99840 0x56000df9e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fda9b7fe640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fda940a2c30 0x7fda940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/1625214755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df99840 msgr2=0x56000df9e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/1625214755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df99840 0x56000df9e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/1625214755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df98e50 msgr2=0x56000df99270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/1625214755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df98e50 0x56000df99270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/1625214755 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56000dc11210 con 0x56000df9e9b0 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.200+0000 7fda9b7fe640 1 -- 172.21.15.110:0/2614434693 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fda9c0cf3f0 con 0x7fdaac104bb0 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.201+0000 7fdaaad9d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fda940a2c30 0x7fda940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.201+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/1625214755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df9e9b0 0x56000dfa0e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0ca6f0 tx=0x7fda9c0668a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1268d9b8f9b8ef03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.201+0000 7fda98ff9640 1 -- 172.21.15.110:0/1625214755 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c09b760 con 0x56000df9e9b0 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.201+0000 7fda98ff9640 1 -- 172.21.15.110:0/1625214755 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda9c0ca0d0 con 0x56000df9e9b0 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.201+0000 7fda98ff9640 1 -- 172.21.15.110:0/1625214755 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00ce60 con 0x56000df9e9b0 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 -- 172.21.15.110:0/1625214755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df9e9b0 msgr2=0x56000dfa0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdaaad9d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fda940a2c30 0x7fda940a50f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00304b0 tx=0x7fdaa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 --2- 172.21.15.110:0/1625214755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df9e9b0 0x56000dfa0e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0ca6f0 tx=0x7fda9c0668a0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 -- 172.21.15.110:0/1625214755 shutdown_connections 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 --2- 172.21.15.110:0/1625214755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df9e9b0 0x56000dfa0e40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 --2- 172.21.15.110:0/1625214755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df99840 0x56000df9e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 --2- 172.21.15.110:0/1625214755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df98e50 0x56000df99270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 -- 172.21.15.110:0/1625214755 >> 172.21.15.110:0/1625214755 conn(0x56000df944c0 msgr2=0x56000df948c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 -- 172.21.15.110:0/1625214755 shutdown_connections 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 -- 172.21.15.110:0/1625214755 wait complete. 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 -- start start 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.202+0000 7fdab3569740 1 Processor -- start 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdab3569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdab3569740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdab3569740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000dfaa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaa59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdab3569740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56000dfa18a0 con 0x56000df99840 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaa59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:44112/0 (socket says 172.21.15.110:44112) 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaa9d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56000deb8b50 con 0x56000df9e9b0 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaa59c640 1 -- 172.21.15.110:0/2215867617 learned_addr learned my addr 172.21.15.110:0/2215867617 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56000dfa1a20 con 0x56000df98e50 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdab3569740 10 client.0 _ll_register_callbacks cb 0x56000def3ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaad9d640 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000dfaa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaa59c640 1 -- 172.21.15.110:0/2215867617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 msgr2=0x56000dfaa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000dfaa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaa59c640 1 -- 172.21.15.110:0/2215867617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 msgr2=0x56000dfa36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaaa59c640 1 -- 172.21.15.110:0/2215867617 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000dcdd620 con 0x56000df98e50 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.203+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa3180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400fdd0 tx=0x7fdaa4004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a16c2f4907a212 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fda91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fda91ffb640 1 -- 172.21.15.110:0/2215867617 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4004b20 con 0x56000df98e50 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fda91ffb640 1 -- 172.21.15.110:0/2215867617 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fdaa4004cc0 con 0x56000df98e50 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56000df0a1f0 con 0x56000df98e50 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fda91ffb640 1 -- 172.21.15.110:0/2215867617 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdaa4005020 con 0x56000df98e50 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fdab3569740 10 client.25741 fetch_fsmap learned FSMap version 32 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56000dfadac0 con 0x56000df98e50 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fda91ffb640 1 -- 172.21.15.110:0/2215867617 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56000df0a1f0 con 0x56000df98e50 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.204+0000 7fda91ffb640 1 -- 172.21.15.110:0/2215867617 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fdaa400b040 con 0x56000df98e50 2024-04-25T00:13:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fdab3569740 10 client.25741 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fdab3569740 10 client.25741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fdab3569740 20 client.25741 populate_metadata read hostname 'smithi110' 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fdab3569740 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56000dfaf200 0x56000dfb1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fdab3569740 4 client.25741 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56000df04540 con 0x56000dfaf200 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56000dfaf200 0x56000dfb1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fda91ffb640 10 client.25741 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.205+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56000dfaf200 0x56000dfb1650 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b2fbed48a7b103 server_cookie=8810f44c01ebd915 in_seq=0 out_seq=0 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fda91ffb640 1 -- 172.21.15.110:0/2215867617 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56000df04540 con 0x56000dfaf200 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fda91ffb640 10 client.25741 handle_command_reply: tid=0 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 client.25741 shutdown 2024-04-25T00:13:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 msgr2=0x56000dfa3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa3180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400fdd0 tx=0x7fdaa4004410 comp rx=0 tx=0).stop 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 shutdown_connections 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56000dfaf200 0x56000dfb1650 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000dfaa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fda91ffb640 0 client.25741 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 --2- 172.21.15.110:0/2215867617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa3180 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.206+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 >> 172.21.15.110:0/2215867617 conn(0x56000df944c0 msgr2=0x56000df935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.207+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 shutdown_connections 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.207+0000 7fdab3569740 1 -- 172.21.15.110:0/2215867617 wait complete. 2024-04-25T00:13:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.207+0000 7fdab3569740 20 client.25741 trim_cache size 0 max 16384 2024-04-25T00:13:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdab3569740 1 Processor -- start 2024-04-25T00:13:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdab3569740 1 -- start start 2024-04-25T00:13:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdab3569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000df99230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdab3569740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfaa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdab3569740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000df9e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdab3569740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1b19b0 con 0x56000df99840 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdab3569740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1af280 con 0x56000df9e9b0 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdab3569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1ad0d0 con 0x56000df98e50 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdaaa59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000df99230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdaa9d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfaa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdaaa59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000df99230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:44124/0 (socket says 172.21.15.110:44124) 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.225+0000 7fdaaa59c640 1 -- 172.21.15.110:0/1534682407 learned_addr learned my addr 172.21.15.110:0/1534682407 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaaad9d640 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000df9e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaaa59c640 1 -- 172.21.15.110:0/1534682407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 msgr2=0x56000df9e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000df9e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaaa59c640 1 -- 172.21.15.110:0/1534682407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 msgr2=0x56000dfaa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfaa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaaa59c640 1 -- 172.21.15.110:0/1534682407 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56000df99c20 con 0x56000df98e50 2024-04-25T00:13:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaaad9d640 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000df9e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfaa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000df99230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4005d50 tx=0x7fdaa40196a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f98bb874a1efaeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fda93fff640 1 -- 172.21.15.110:0/1534682407 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40096b0 con 0x56000df98e50 2024-04-25T00:13:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fda93fff640 1 -- 172.21.15.110:0/1534682407 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa4004dc0 con 0x56000df98e50 2024-04-25T00:13:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.226+0000 7fda93fff640 1 -- 172.21.15.110:0/1534682407 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4005870 con 0x56000df98e50 2024-04-25T00:13:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 -- 172.21.15.110:0/1534682407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 msgr2=0x56000df99230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000df99230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4005d50 tx=0x7fdaa40196a0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 -- 172.21.15.110:0/1534682407 shutdown_connections 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000df9e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfaa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 --2- 172.21.15.110:0/1534682407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000df99230 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 -- 172.21.15.110:0/1534682407 >> 172.21.15.110:0/1534682407 conn(0x56000df944c0 msgr2=0x56000df9a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 -- 172.21.15.110:0/1534682407 shutdown_connections 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.227+0000 7fdab3569740 1 -- 172.21.15.110:0/1534682407 wait complete. 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 1 -- start start 2024-04-25T00:13:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 1 Processor -- start 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000e0137c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1ad740 con 0x56000df99840 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1aebb0 con 0x56000df9e9b0 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1ade10 con 0x56000df98e50 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdab3569740 10 client.0 _ll_register_callbacks cb 0x7fdaa4009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdaa9d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdaaa59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdaa9d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40268/0 (socket says 172.21.15.110:40268) 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/160314357 learned_addr learned my addr 172.21.15.110:0/160314357 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.228+0000 7fdaaad9d640 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000e0137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/160314357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 msgr2=0x56000dfa7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/160314357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 msgr2=0x56000e0137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000e0137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fdaa9d9b640 1 -- 172.21.15.110:0/160314357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000e013d00 con 0x56000df99840 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fdaa9d9b640 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa7b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0985a0 tx=0x7fda9c0048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73dce3fd06861d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fda91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fda91ffb640 1 -- 172.21.15.110:0/160314357 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c09b580 con 0x56000df99840 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fda91ffb640 1 -- 172.21.15.110:0/160314357 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fda9c099cc0 con 0x56000df99840 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fdab3569740 1 -- 172.21.15.110:0/160314357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa4004dc0 con 0x56000df99840 2024-04-25T00:13:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.229+0000 7fda91ffb640 1 -- 172.21.15.110:0/160314357 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda9c005310 con 0x56000df99840 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdab3569740 10 client.16176 fetch_fsmap learned FSMap version 32 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdab3569740 1 -- 172.21.15.110:0/160314357 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56000e014fc0 con 0x56000df99840 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fda91ffb640 1 -- 172.21.15.110:0/160314357 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaa4004dc0 con 0x56000df99840 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fda91ffb640 1 -- 172.21.15.110:0/160314357 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fda9c0ccf40 con 0x56000df99840 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdab3569740 10 client.16176 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdab3569740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdab3569740 20 client.16176 populate_metadata read hostname 'smithi110' 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdab3569740 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56000e0155a0 0x56000e017990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdab3569740 4 client.16176 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:13:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdab3569740 1 -- 172.21.15.110:0/160314357 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56000de828e0 con 0x56000e0155a0 2024-04-25T00:13:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.230+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56000e0155a0 0x56000e017990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.231+0000 7fda91ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.231+0000 7fdaaa59c640 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56000e0155a0 0x56000e017990 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb3ad78b3b517e11 server_cookie=a0f640348a0c1ef9 in_seq=0 out_seq=0 2024-04-25T00:13:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fda91ffb640 1 -- 172.21.15.110:0/160314357 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56000de828e0 con 0x56000e0155a0 2024-04-25T00:13:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fda91ffb640 10 client.16176 handle_command_reply: tid=0 2024-04-25T00:13:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 client.16176 shutdown 2024-04-25T00:13:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 -- 172.21.15.110:0/160314357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 msgr2=0x56000dfa7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa7b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0985a0 tx=0x7fda9c0048b0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 -- 172.21.15.110:0/160314357 shutdown_connections 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x56000e0155a0 0x56000e017990 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56000df9e9b0 0x56000e0137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fda91ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56000df99840 0x56000dfa7b50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 --2- 172.21.15.110:0/160314357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000df98e50 0x56000dfa7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 -- 172.21.15.110:0/160314357 >> 172.21.15.110:0/160314357 conn(0x56000df944c0 msgr2=0x56000df9e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 -- 172.21.15.110:0/160314357 shutdown_connections 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 1 -- 172.21.15.110:0/160314357 wait complete. 2024-04-25T00:13:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.254+0000 7fdab3569740 20 client.16176 trim_cache size 0 max 16384 2024-04-25T00:13:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:13:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:13:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:13:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 -- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fda940a2c30 msgr2=0x7fda940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fda940a2c30 0x7fda940a50f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00304b0 tx=0x7fdaa0005f90 comp rx=0 tx=0).stop 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 -- 172.21.15.110:0/2614434693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac104bb0 msgr2=0x7fdaac196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac104bb0 0x7fdaac196880 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00e9f0 tx=0x7fda9c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 -- 172.21.15.110:0/2614434693 shutdown_connections 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fda940a2c30 0x7fda940a50f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaac1074e0 0x7fdaac192eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaac104bb0 0x7fdaac196880 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 --2- 172.21.15.110:0/2614434693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaac102280 0x7fdaac196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 -- 172.21.15.110:0/2614434693 >> 172.21.15.110:0/2614434693 conn(0x7fdaac0fbc10 msgr2=0x7fdaac101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 -- 172.21.15.110:0/2614434693 shutdown_connections 2024-04-25T00:13:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:13:47.255+0000 7fdab085d640 1 -- 172.21.15.110:0/2614434693 wait complete. 2024-04-25T00:13:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:47 smithi173 ceph-mon[32402]: pgmap v703: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.4 MiB/s wr, 898 op/s 2024-04-25T00:13:48.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:47 smithi188 ceph-mon[33285]: pgmap v703: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.4 MiB/s wr, 898 op/s 2024-04-25T00:13:48.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:47 smithi110 ceph-mon[29187]: pgmap v703: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.4 MiB/s wr, 898 op/s 2024-04-25T00:13:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:48 smithi188 ceph-mon[33285]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:13:49.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:48 smithi110 ceph-mon[29187]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:13:49.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:48 smithi173 ceph-mon[32402]: pgmap v704: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-04-25T00:13:50.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:50 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:50.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:50 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:50.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:50 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 645.0 s, 801.8 tps, lat 124.257 ms stddev 23.416 2024-04-25T00:13:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:51.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:51.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:51.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:51 smithi173 ceph-mon[32402]: pgmap v705: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 917 op/s 2024-04-25T00:13:51.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:51.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:51 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:51 smithi188 ceph-mon[33285]: pgmap v705: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 917 op/s 2024-04-25T00:13:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:51 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:51 smithi110 ceph-mon[29187]: pgmap v705: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 917 op/s 2024-04-25T00:13:52.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:52.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:51 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:53.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:53.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:53.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:53.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:13:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:53 smithi188 ceph-mon[33285]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-04-25T00:13:54.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:53 smithi110 ceph-mon[29187]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-04-25T00:13:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:53 smithi173 ceph-mon[32402]: pgmap v706: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-04-25T00:13:56.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:55 smithi110 ceph-mon[29187]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.1 MiB/s wr, 906 op/s 2024-04-25T00:13:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:55 smithi173 ceph-mon[32402]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.1 MiB/s wr, 906 op/s 2024-04-25T00:13:56.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:55 smithi188 ceph-mon[33285]: pgmap v707: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.1 MiB/s wr, 906 op/s 2024-04-25T00:13:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 650.0 s, 831.4 tps, lat 120.468 ms stddev 18.767 2024-04-25T00:13:58.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:57 smithi110 ceph-mon[29187]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.9 MiB/s wr, 872 op/s 2024-04-25T00:13:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:57 smithi173 ceph-mon[32402]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.9 MiB/s wr, 872 op/s 2024-04-25T00:13:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:57 smithi188 ceph-mon[33285]: pgmap v708: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.9 MiB/s wr, 872 op/s 2024-04-25T00:14:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:13:59 smithi173 ceph-mon[32402]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-04-25T00:14:00.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:13:59 smithi188 ceph-mon[33285]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-04-25T00:14:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:13:59 smithi110 ceph-mon[29187]: pgmap v709: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 -- 172.21.15.110:0/1480696803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 msgr2=0x7f021c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 --2- 172.21.15.110:0/1480696803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 0x7f021c101d10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0210009900 tx=0x7f021002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 -- 172.21.15.110:0/1480696803 shutdown_connections 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 --2- 172.21.15.110:0/1480696803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f021c102250 0x7f021c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 --2- 172.21.15.110:0/1480696803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 0x7f021c101d10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 --2- 172.21.15.110:0/1480696803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c1006a0 0x7f021c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 -- 172.21.15.110:0/1480696803 >> 172.21.15.110:0/1480696803 conn(0x7f021c0fbc10 msgr2=0x7f021c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 -- 172.21.15.110:0/1480696803 shutdown_connections 2024-04-25T00:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.366+0000 7f02212fc640 1 -- 172.21.15.110:0/1480696803 wait complete. 2024-04-25T00:14:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f02212fc640 1 Processor -- start 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f02212fc640 1 -- start start 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f02212fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f021c1006a0 0x7f021c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f02212fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 0x7f021c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f02212fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c102250 0x7f021c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f02212fc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10c1d0 con 0x7f021c101890 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f02212fc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10c050 con 0x7f021c1006a0 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f02212fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10c350 con 0x7f021c102250 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 0x7f021c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f021c1006a0 0x7f021c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 0x7f021c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54970/0 (socket says 172.21.15.110:54970) 2024-04-25T00:14:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021a7fc640 1 -- 172.21.15.110:0/3467613558 learned_addr learned my addr 172.21.15.110:0/3467613558 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021b7fe640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c102250 0x7f021c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021a7fc640 1 -- 172.21.15.110:0/3467613558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c102250 msgr2=0x7f021c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021a7fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c102250 0x7f021c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021a7fc640 1 -- 172.21.15.110:0/3467613558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f021c1006a0 msgr2=0x7f021c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021a7fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f021c1006a0 0x7f021c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021a7fc640 1 -- 172.21.15.110:0/3467613558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c19f5a0 con 0x7f021c101890 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.367+0000 7f021affd640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f021c1006a0 0x7f021c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.368+0000 7f021a7fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 0x7f021c197f10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0210005940 tx=0x7f02100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.368+0000 7f0203fff640 1 -- 172.21.15.110:0/3467613558 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021003f030 con 0x7f021c101890 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.368+0000 7f02212fc640 1 -- 172.21.15.110:0/3467613558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c19f830 con 0x7f021c101890 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.368+0000 7f02212fc640 1 -- 172.21.15.110:0/3467613558 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c19fd70 con 0x7f021c101890 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.368+0000 7f0203fff640 1 -- 172.21.15.110:0/3467613558 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02100318f0 con 0x7f021c101890 2024-04-25T00:14:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.369+0000 7f0203fff640 1 -- 172.21.15.110:0/3467613558 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210035070 con 0x7f021c101890 2024-04-25T00:14:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.371+0000 7f0203fff640 1 -- 172.21.15.110:0/3467613558 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0210042030 con 0x7f021c101890 2024-04-25T00:14:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.371+0000 7f0224008740 1 Processor -- start 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.371+0000 7f0224008740 1 -- start start 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f0224008740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd290270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f0224008740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd290840 0x5574dd295340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f0224008740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd2959b0 0x5574dd297e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f0224008740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574dd1f3cb0 con 0x5574dd290840 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f0224008740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5574dd19c5b0 con 0x5574dd28fe50 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f0224008740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5574dd1efc70 con 0x5574dd2959b0 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd290270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd290840 0x5574dd295340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021a7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd2959b0 0x5574dd297e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd290270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43832/0 (socket says 172.21.15.110:43832) 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 -- 172.21.15.110:0/510946366 learned_addr learned my addr 172.21.15.110:0/510946366 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f0203fff640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02040a2cb0 0x7f02040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 -- 172.21.15.110:0/510946366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd2959b0 msgr2=0x5574dd297e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 --2- 172.21.15.110:0/510946366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd2959b0 0x5574dd297e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 -- 172.21.15.110:0/510946366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd290840 msgr2=0x5574dd295340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 --2- 172.21.15.110:0/510946366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd290840 0x5574dd295340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 -- 172.21.15.110:0/510946366 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574dcf08210 con 0x5574dd28fe50 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f0203fff640 1 -- 172.21.15.110:0/3467613558 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f02100ef440 con 0x7f021c101890 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.372+0000 7f021b7fe640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02040a2cb0 0x7f02040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.373+0000 7f021b7fe640 1 --2- 172.21.15.110:0/510946366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd290270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0214013e70 tx=0x7f0214009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b67cf8dab381739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.373+0000 7f02017fa640 1 -- 172.21.15.110:0/510946366 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02140035f0 con 0x5574dd28fe50 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.373+0000 7f02017fa640 1 -- 172.21.15.110:0/510946366 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0214003c40 con 0x5574dd28fe50 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.373+0000 7f021b7fe640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02040a2cb0 0x7f02040a5170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0214026000 tx=0x7f0214023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.373+0000 7f02017fa640 1 -- 172.21.15.110:0/510946366 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214029940 con 0x5574dd28fe50 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 -- 172.21.15.110:0/510946366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 msgr2=0x5574dd290270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 --2- 172.21.15.110:0/510946366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd290270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0214013e70 tx=0x7f0214009b50 comp rx=0 tx=0).stop 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 -- 172.21.15.110:0/510946366 shutdown_connections 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 --2- 172.21.15.110:0/510946366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd2959b0 0x5574dd297e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 --2- 172.21.15.110:0/510946366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd290840 0x5574dd295340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 --2- 172.21.15.110:0/510946366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd290270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 -- 172.21.15.110:0/510946366 >> 172.21.15.110:0/510946366 conn(0x5574dd28b4c0 msgr2=0x5574dd28b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 -- 172.21.15.110:0/510946366 shutdown_connections 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 -- 172.21.15.110:0/510946366 wait complete. 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 -- start start 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.374+0000 7f0224008740 1 Processor -- start 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f0224008740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd2a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f0224008740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f0224008740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f0224008740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574dd298960 con 0x5574dd2959b0 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f0224008740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5574dd1afb50 con 0x5574dd28fe50 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f0224008740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5574dd298ae0 con 0x5574dd290840 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f0224008740 10 client.0 _ll_register_callbacks cb 0x5574dd1eaca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd2a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021a7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd2a2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43844/0 (socket says 172.21.15.110:43844) 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 -- 172.21.15.110:0/3614090830 learned_addr learned my addr 172.21.15.110:0/3614090830 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021b7fe640 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 -- 172.21.15.110:0/3614090830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 msgr2=0x5574dd2a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 -- 172.21.15.110:0/3614090830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 msgr2=0x5574dd2aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 -- 172.21.15.110:0/3614090830 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574dcfd4620 con 0x5574dd28fe50 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021b7fe640 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f021affd640 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd2a2b10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f020c005fd0 tx=0x7f020c012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e1699bea36b9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.375+0000 7f01feff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f01feff5640 1 -- 172.21.15.110:0/3614090830 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c003970 con 0x5574dd28fe50 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 1 -- 172.21.15.110:0/3614090830 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0214003c40 con 0x5574dd28fe50 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f01feff5640 1 -- 172.21.15.110:0/3614090830 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f020c003b10 con 0x5574dd28fe50 2024-04-25T00:14:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f01feff5640 1 -- 172.21.15.110:0/3614090830 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f020c023070 con 0x5574dd28fe50 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 10 client.25778 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 1 -- 172.21.15.110:0/3614090830 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5574dd2ad470 con 0x5574dd28fe50 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f01feff5640 1 -- 172.21.15.110:0/3614090830 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0214003c40 con 0x5574dd28fe50 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f01feff5640 1 -- 172.21.15.110:0/3614090830 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f020c010430 con 0x5574dd28fe50 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 10 client.25778 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 10 client.25778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 20 client.25778 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574dd2aef70 0x5574dd2b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 4 client.25778 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.376+0000 7f0224008740 1 -- 172.21.15.110:0/3614090830 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574dd2011f0 con 0x5574dd2aef70 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.377+0000 7f021a7fc640 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574dd2aef70 0x5574dd2b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.377+0000 7f021a7fc640 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574dd2aef70 0x5574dd2b1410 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62a04f208affb96 server_cookie=b5e58f760cc7a1e1 in_seq=0 out_seq=0 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.377+0000 7f01feff5640 10 client.25778 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.377+0000 7f01feff5640 1 -- 172.21.15.110:0/3614090830 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574dd2011f0 con 0x5574dd2aef70 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.377+0000 7f01feff5640 10 client.25778 handle_command_reply: tid=0 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.377+0000 7f0224008740 1 client.25778 shutdown 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 -- 172.21.15.110:0/3614090830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 msgr2=0x5574dd2a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd2a2b10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f020c005fd0 tx=0x7f020c012820 comp rx=0 tx=0).stop 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 -- 172.21.15.110:0/3614090830 shutdown_connections 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574dd2aef70 0x5574dd2b1410 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 --2- 172.21.15.110:0/3614090830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd28fe50 0x5574dd2a2b10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 -- 172.21.15.110:0/3614090830 >> 172.21.15.110:0/3614090830 conn(0x5574dd28b4c0 msgr2=0x5574dd28a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f01feff5640 0 client.25778 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 -- 172.21.15.110:0/3614090830 shutdown_connections 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 1 -- 172.21.15.110:0/3614090830 wait complete. 2024-04-25T00:14:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.378+0000 7f0224008740 20 client.25778 trim_cache size 0 max 16384 2024-04-25T00:14:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.398+0000 7f0224008740 1 Processor -- start 2024-04-25T00:14:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.398+0000 7f0224008740 1 -- start start 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.398+0000 7f0224008740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd28fe50 0x5574dd291ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.398+0000 7f0224008740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.398+0000 7f0224008740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd2959b0 0x5574dd2aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.398+0000 7f0224008740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1a9160 con 0x5574dd28fe50 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.398+0000 7f0224008740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1a6a30 con 0x5574dd2959b0 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.398+0000 7f0224008740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1a48a0 con 0x5574dd290840 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd28fe50 0x5574dd291ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd28fe50 0x5574dd291ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55000/0 (socket says 172.21.15.110:55000) 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 -- 172.21.15.110:0/3672594807 learned_addr learned my addr 172.21.15.110:0/3672594807 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021a7fc640 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021b7fe640 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd2959b0 0x5574dd2aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 -- 172.21.15.110:0/3672594807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 msgr2=0x5574dd2921e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2921e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 -- 172.21.15.110:0/3672594807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd2959b0 msgr2=0x5574dd2aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd2959b0 0x5574dd2aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 -- 172.21.15.110:0/3672594807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574dd297da0 con 0x5574dd28fe50 2024-04-25T00:14:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021a7fc640 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021b7fe640 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd2959b0 0x5574dd2aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.399+0000 7f021affd640 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd28fe50 0x5574dd291ca0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f020c014000 tx=0x7f020c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cfb0cd3582ac57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.400+0000 7f0200ff9640 1 -- 172.21.15.110:0/3672594807 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c010430 con 0x5574dd28fe50 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.400+0000 7f0200ff9640 1 -- 172.21.15.110:0/3672594807 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f020c0095b0 con 0x5574dd28fe50 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.400+0000 7f0200ff9640 1 -- 172.21.15.110:0/3672594807 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c005460 con 0x5574dd28fe50 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 -- 172.21.15.110:0/3672594807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd28fe50 msgr2=0x5574dd291ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd28fe50 0x5574dd291ca0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f020c014000 tx=0x7f020c010da0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 -- 172.21.15.110:0/3672594807 shutdown_connections 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd2959b0 0x5574dd2aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd290840 0x5574dd2921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 --2- 172.21.15.110:0/3672594807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd28fe50 0x5574dd291ca0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 -- 172.21.15.110:0/3672594807 >> 172.21.15.110:0/3672594807 conn(0x5574dd28b4c0 msgr2=0x5574dd28b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 -- 172.21.15.110:0/3672594807 shutdown_connections 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 -- 172.21.15.110:0/3672594807 wait complete. 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.401+0000 7f0224008740 1 -- start start 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f0224008740 1 Processor -- start 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f0224008740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd290840 0x5574dd2a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f0224008740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2a7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f0224008740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd311570 0x5574dd313960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f0224008740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1a4ef0 con 0x5574dd2959b0 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f0224008740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1a6360 con 0x5574dd290840 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f0224008740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1a55c0 con 0x5574dd311570 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f0224008740 10 client.0 _ll_register_callbacks cb 0x7f020c00f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021a7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2a7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55014/0 (socket says 172.21.15.110:55014) 2024-04-25T00:14:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021a7fc640 1 -- 172.21.15.110:0/2508743889 learned_addr learned my addr 172.21.15.110:0/2508743889 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021b7fe640 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd311570 0x5574dd313960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021affd640 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd290840 0x5574dd2a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021a7fc640 1 -- 172.21.15.110:0/2508743889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd311570 msgr2=0x5574dd313960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021a7fc640 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd311570 0x5574dd313960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021a7fc640 1 -- 172.21.15.110:0/2508743889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd290840 msgr2=0x5574dd2a6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021a7fc640 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd290840 0x5574dd2a6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.402+0000 7f021a7fc640 1 -- 172.21.15.110:0/2508743889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574dd313ea0 con 0x5574dd2959b0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.403+0000 7f021a7fc640 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2a7390 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f02100b8750 tx=0x7f02100bb8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4a62d2121518bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.403+0000 7f01feff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.403+0000 7f01feff5640 1 -- 172.21.15.110:0/2508743889 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210049040 con 0x5574dd2959b0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.403+0000 7f01feff5640 1 -- 172.21.15.110:0/2508743889 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f02100f2cf0 con 0x5574dd2959b0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.403+0000 7f0224008740 1 -- 172.21.15.110:0/2508743889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f020c0095b0 con 0x5574dd2959b0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.403+0000 7f01feff5640 1 -- 172.21.15.110:0/2508743889 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f02100d9050 con 0x5574dd2959b0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.403+0000 7f0224008740 10 client.16194 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.403+0000 7f0224008740 1 -- 172.21.15.110:0/2508743889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5574dd315140 con 0x5574dd2959b0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f01feff5640 1 -- 172.21.15.110:0/2508743889 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f020c0095b0 con 0x5574dd2959b0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f01feff5640 1 -- 172.21.15.110:0/2508743889 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f02100ef0c0 con 0x5574dd2959b0 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f0224008740 10 client.16194 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f0224008740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f0224008740 20 client.16194 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f0224008740 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574dd315a80 0x5574dd317ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f0224008740 4 client.16194 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:14:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f0224008740 1 -- 172.21.15.110:0/2508743889 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574dd2917b0 con 0x5574dd315a80 2024-04-25T00:14:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f021affd640 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574dd315a80 0x5574dd317ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.404+0000 7f01feff5640 10 client.16194 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.405+0000 7f021affd640 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574dd315a80 0x5574dd317ea0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bed01f75a859cef server_cookie=c5a3c5d3ef960680 in_seq=0 out_seq=0 2024-04-25T00:14:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f01feff5640 1 -- 172.21.15.110:0/2508743889 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574dd2917b0 con 0x5574dd315a80 2024-04-25T00:14:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f01feff5640 10 client.16194 handle_command_reply: tid=0 2024-04-25T00:14:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 client.16194 shutdown 2024-04-25T00:14:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 -- 172.21.15.110:0/2508743889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 msgr2=0x5574dd2a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2a7390 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f02100b8750 tx=0x7f02100bb8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 -- 172.21.15.110:0/2508743889 shutdown_connections 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5574dd315a80 0x5574dd317ea0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f01feff5640 0 client.16194 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574dd311570 0x5574dd313960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574dd2959b0 0x5574dd2a7390 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 --2- 172.21.15.110:0/2508743889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5574dd290840 0x5574dd2a6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.425+0000 7f0224008740 1 -- 172.21.15.110:0/2508743889 >> 172.21.15.110:0/2508743889 conn(0x5574dd28b4c0 msgr2=0x5574dd291ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.426+0000 7f0224008740 1 -- 172.21.15.110:0/2508743889 shutdown_connections 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.426+0000 7f0224008740 1 -- 172.21.15.110:0/2508743889 wait complete. 2024-04-25T00:14:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.426+0000 7f0224008740 20 client.16194 trim_cache size 0 max 16384 2024-04-25T00:14:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:14:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:14:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:14:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 -- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02040a2cb0 msgr2=0x7f02040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02040a2cb0 0x7f02040a5170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0214026000 tx=0x7f0214023000 comp rx=0 tx=0).stop 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 -- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 msgr2=0x7f021c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 0x7f021c197f10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0210005940 tx=0x7f02100042f0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 -- 172.21.15.110:0/3467613558 shutdown_connections 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f02040a2cb0 0x7f02040a5170 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c102250 0x7f021c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f021c101890 0x7f021c197f10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 --2- 172.21.15.110:0/3467613558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f021c1006a0 0x7f021c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 -- 172.21.15.110:0/3467613558 >> 172.21.15.110:0/3467613558 conn(0x7f021c0fbc10 msgr2=0x7f021c0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 -- 172.21.15.110:0/3467613558 shutdown_connections 2024-04-25T00:14:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:00.427+0000 7f02212fc640 1 -- 172.21.15.110:0/3467613558 wait complete. 2024-04-25T00:14:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 655.0 s, 820.4 tps, lat 122.000 ms stddev 14.507 2024-04-25T00:14:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:01 smithi173 ceph-mon[32402]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.9 MiB/s wr, 879 op/s 2024-04-25T00:14:02.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:01 smithi188 ceph-mon[33285]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.9 MiB/s wr, 879 op/s 2024-04-25T00:14:02.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:01 smithi110 ceph-mon[29187]: pgmap v710: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.9 MiB/s wr, 879 op/s 2024-04-25T00:14:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:03 smithi173 ceph-mon[32402]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-04-25T00:14:04.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:03 smithi188 ceph-mon[33285]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-04-25T00:14:04.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:03 smithi110 ceph-mon[29187]: pgmap v711: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 10 MiB/s wr, 1.07k op/s 2024-04-25T00:14:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:05 smithi173 ceph-mon[32402]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.0 MiB/s wr, 842 op/s 2024-04-25T00:14:06.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:05 smithi188 ceph-mon[33285]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.0 MiB/s wr, 842 op/s 2024-04-25T00:14:06.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:05 smithi110 ceph-mon[29187]: pgmap v712: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.0 MiB/s wr, 842 op/s 2024-04-25T00:14:06.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 660.0 s, 816.4 tps, lat 122.044 ms stddev 16.489 2024-04-25T00:14:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:07 smithi173 ceph-mon[32402]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 897 op/s 2024-04-25T00:14:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:07 smithi188 ceph-mon[33285]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 897 op/s 2024-04-25T00:14:08.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:07 smithi110 ceph-mon[29187]: pgmap v713: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 897 op/s 2024-04-25T00:14:10.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:09 smithi188 ceph-mon[33285]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.9 MiB/s wr, 1.04k op/s 2024-04-25T00:14:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:09 smithi173 ceph-mon[32402]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.9 MiB/s wr, 1.04k op/s 2024-04-25T00:14:10.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:09 smithi110 ceph-mon[29187]: pgmap v714: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.9 MiB/s wr, 1.04k op/s 2024-04-25T00:14:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 665.0 s, 832.4 tps, lat 120.726 ms stddev 23.709 2024-04-25T00:14:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:11 smithi173 ceph-mon[32402]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 854 op/s 2024-04-25T00:14:12.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:11 smithi188 ceph-mon[33285]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 854 op/s 2024-04-25T00:14:12.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:11 smithi110 ceph-mon[29187]: pgmap v715: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 854 op/s 2024-04-25T00:14:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.557+0000 7fc8909af640 1 -- 172.21.15.110:0/611711273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c1090a0 msgr2=0x7fc88c10fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.557+0000 7fc8909af640 1 --2- 172.21.15.110:0/611711273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c1090a0 0x7fc88c10fa80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc884009ca0 tx=0x7fc88402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.558+0000 7fc8909af640 1 -- 172.21.15.110:0/611711273 shutdown_connections 2024-04-25T00:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.558+0000 7fc8909af640 1 --2- 172.21.15.110:0/611711273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88c10ffc0 0x7fc88c1123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.558+0000 7fc8909af640 1 --2- 172.21.15.110:0/611711273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c1090a0 0x7fc88c10fa80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.558+0000 7fc8909af640 1 --2- 172.21.15.110:0/611711273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c108740 0x7fc88c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.558+0000 7fc8909af640 1 -- 172.21.15.110:0/611711273 >> 172.21.15.110:0/611711273 conn(0x7fc88c06e7b0 msgr2=0x7fc88c070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.558+0000 7fc8909af640 1 -- 172.21.15.110:0/611711273 shutdown_connections 2024-04-25T00:14:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.558+0000 7fc8909af640 1 -- 172.21.15.110:0/611711273 wait complete. 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc8909af640 1 Processor -- start 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc8909af640 1 -- start start 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc8909af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c108740 0x7fc88c1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc8909af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88c1090a0 0x7fc88c1a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc8909af640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c10ffc0 0x7fc88c1a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc8909af640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10ab50 con 0x7fc88c1090a0 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc8909af640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10a9d0 con 0x7fc88c10ffc0 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc8909af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10acd0 con 0x7fc88c108740 2024-04-25T00:14:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc88ad9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c10ffc0 0x7fc88c1a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88c1090a0 0x7fc88c1a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc88ad9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c10ffc0 0x7fc88c1a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:37494/0 (socket says 172.21.15.110:37494) 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc88a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c108740 0x7fc88c1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc88ad9d640 1 -- 172.21.15.110:0/2654851743 learned_addr learned my addr 172.21.15.110:0/2654851743 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc88ad9d640 1 -- 172.21.15.110:0/2654851743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c108740 msgr2=0x7fc88c1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.559+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c108740 0x7fc88c1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc88ad9d640 1 -- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88c1090a0 msgr2=0x7fc88c1a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88c1090a0 0x7fc88c1a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc88ad9d640 1 -- 172.21.15.110:0/2654851743 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c1a9ec0 con 0x7fc88c10ffc0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c10ffc0 0x7fc88c1a9850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc88000e9b0 tx=0x7fc88000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc87b7fe640 1 -- 172.21.15.110:0/2654851743 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88000cd80 con 0x7fc88c10ffc0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc87b7fe640 1 -- 172.21.15.110:0/2654851743 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8800035f0 con 0x7fc88c10ffc0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc8909af640 1 -- 172.21.15.110:0/2654851743 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c1aa1b0 con 0x7fc88c10ffc0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc87b7fe640 1 -- 172.21.15.110:0/2654851743 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880010690 con 0x7fc88c10ffc0 2024-04-25T00:14:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.560+0000 7fc8909af640 1 -- 172.21.15.110:0/2654851743 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c1aa700 con 0x7fc88c10ffc0 2024-04-25T00:14:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.562+0000 7fc87b7fe640 1 -- 172.21.15.110:0/2654851743 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc8800037a0 con 0x7fc88c10ffc0 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.563+0000 7fc893795740 1 Processor -- start 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.563+0000 7fc87b7fe640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8740a2cb0 0x7fc8740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.563+0000 7fc889d9b640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8740a2cb0 0x7fc8740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.563+0000 7fc893795740 1 -- start start 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.563+0000 7fc893795740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f1d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.563+0000 7fc893795740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f22340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.563+0000 7fc893795740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f24e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc893795740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e6b4e80cb0 con 0x55e6b4f1ce50 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc893795740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e6b4e295b0 con 0x55e6b4f1d840 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc893795740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e6b4e7cc70 con 0x55e6b4f229b0 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc87b7fe640 1 -- 172.21.15.110:0/2654851743 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc8800cf2b0 con 0x7fc88c10ffc0 2024-04-25T00:14:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f24e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f1d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f24e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:36942/0 (socket says 172.21.15.110:36942) 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 -- 172.21.15.110:0/2027969682 learned_addr learned my addr 172.21.15.110:0/2027969682 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc889d9b640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8740a2cb0 0x7fc8740a5170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc88c1a37b0 tx=0x7fc884000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 -- 172.21.15.110:0/2027969682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 msgr2=0x55e6b4f22340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2027969682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f22340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 -- 172.21.15.110:0/2027969682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 msgr2=0x55e6b4f1d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2027969682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f1d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 -- 172.21.15.110:0/2027969682 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6b4b95210 con 0x55e6b4f229b0 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2027969682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f24e40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc880097fa0 tx=0x7fc88009b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c93a1abee67ed71c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc878ff9640 1 -- 172.21.15.110:0/2027969682 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88009c240 con 0x55e6b4f229b0 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.564+0000 7fc878ff9640 1 -- 172.21.15.110:0/2027969682 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc88001fd70 con 0x55e6b4f229b0 2024-04-25T00:14:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.565+0000 7fc878ff9640 1 -- 172.21.15.110:0/2027969682 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880099c70 con 0x55e6b4f229b0 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.565+0000 7fc893795740 1 -- 172.21.15.110:0/2027969682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 msgr2=0x55e6b4f24e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.565+0000 7fc893795740 1 --2- 172.21.15.110:0/2027969682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f24e40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc880097fa0 tx=0x7fc88009b4a0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.565+0000 7fc893795740 1 -- 172.21.15.110:0/2027969682 shutdown_connections 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.565+0000 7fc893795740 1 --2- 172.21.15.110:0/2027969682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f24e40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.565+0000 7fc893795740 1 --2- 172.21.15.110:0/2027969682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f22340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.565+0000 7fc893795740 1 --2- 172.21.15.110:0/2027969682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f1d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.566+0000 7fc893795740 1 -- 172.21.15.110:0/2027969682 >> 172.21.15.110:0/2027969682 conn(0x55e6b4f184c0 msgr2=0x55e6b4f188c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.566+0000 7fc893795740 1 -- 172.21.15.110:0/2027969682 shutdown_connections 2024-04-25T00:14:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.566+0000 7fc893795740 1 -- 172.21.15.110:0/2027969682 wait complete. 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.566+0000 7fc893795740 1 -- start start 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.566+0000 7fc893795740 1 Processor -- start 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc893795740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f27360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc893795740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1d840 0x55e6b4f278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc893795740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1d840 0x55e6b4f278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88a59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f27360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1d840 0x55e6b4f278a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38010/0 (socket says 172.21.15.110:38010) 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88a59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f27360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:37518/0 (socket says 172.21.15.110:37518) 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88a59c640 1 -- 172.21.15.110:0/2240398308 learned_addr learned my addr 172.21.15.110:0/2240398308 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e6b4f25960 con 0x55e6b4f1d840 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e6b4e3cb50 con 0x55e6b4f1ce50 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e6b4f25ae0 con 0x55e6b4f229b0 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc893795740 10 client.0 _ll_register_callbacks cb 0x55e6b4e77ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88a59c640 1 -- 172.21.15.110:0/2240398308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 msgr2=0x55e6b4f2e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88a59c640 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88a59c640 1 -- 172.21.15.110:0/2240398308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1d840 msgr2=0x55e6b4f278a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88a59c640 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1d840 0x55e6b4f278a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88a59c640 1 -- 172.21.15.110:0/2240398308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6b4c61620 con 0x55e6b4f1ce50 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.567+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc88a59c640 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f27360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc87c012720 tx=0x7fc87c012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454435a66d2d0347 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc871ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc871ffb640 1 -- 172.21.15.110:0/2240398308 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00f920 con 0x55e6b4f1ce50 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc88001fd70 con 0x55e6b4f1ce50 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc871ffb640 1 -- 172.21.15.110:0/2240398308 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc87c00fac0 con 0x55e6b4f1ce50 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc871ffb640 1 -- 172.21.15.110:0/2240398308 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc87c00fd00 con 0x55e6b4f1ce50 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 10 client.25793 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6b4f31ca0 con 0x55e6b4f1ce50 2024-04-25T00:14:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc871ffb640 1 -- 172.21.15.110:0/2240398308 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc88001fd70 con 0x55e6b4f1ce50 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc871ffb640 1 -- 172.21.15.110:0/2240398308 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc87c010430 con 0x55e6b4f1ce50 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 10 client.25793 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 10 client.25793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 20 client.25793 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e6b4f33330 0x55e6b4f35780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 4 client.25793 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.568+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6b4e8e1f0 con 0x55e6b4f33330 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.569+0000 7fc889d9b640 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e6b4f33330 0x55e6b4f35780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.569+0000 7fc889d9b640 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e6b4f33330 0x55e6b4f35780 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac891e200a0fd36b server_cookie=1b6d5c57e3de1739 in_seq=0 out_seq=0 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.569+0000 7fc871ffb640 10 client.25793 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc871ffb640 1 -- 172.21.15.110:0/2240398308 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6b4e8e1f0 con 0x55e6b4f33330 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc871ffb640 10 client.25793 handle_command_reply: tid=0 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 client.25793 shutdown 2024-04-25T00:14:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1ce50 msgr2=0x55e6b4f27360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f27360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc87c012720 tx=0x7fc87c012be0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 shutdown_connections 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e6b4f33330 0x55e6b4f35780 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc871ffb640 0 client.25793 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1d840 0x55e6b4f278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 --2- 172.21.15.110:0/2240398308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f27360 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 >> 172.21.15.110:0/2240398308 conn(0x55e6b4f184c0 msgr2=0x55e6b4f175c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 shutdown_connections 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.570+0000 7fc893795740 1 -- 172.21.15.110:0/2240398308 wait complete. 2024-04-25T00:14:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.571+0000 7fc893795740 20 client.25793 trim_cache size 0 max 16384 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.590+0000 7fc893795740 1 Processor -- start 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc893795740 1 -- start start 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc893795740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f1bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc893795740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc893795740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f26980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc893795740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1b5d10 con 0x55e6b4f1ce50 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc893795740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1b35e0 con 0x55e6b4f1d840 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc893795740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1b1430 con 0x55e6b4f229b0 2024-04-25T00:14:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:37534/0 (socket says 172.21.15.110:37534) 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc889d9b640 1 -- 172.21.15.110:0/4141177074 learned_addr learned my addr 172.21.15.110:0/4141177074 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc88a59c640 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f1bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f26980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc889d9b640 1 -- 172.21.15.110:0/4141177074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 msgr2=0x55e6b4f26980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc889d9b640 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f26980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc889d9b640 1 -- 172.21.15.110:0/4141177074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 msgr2=0x55e6b4f1bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc889d9b640 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f1bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.591+0000 7fc889d9b640 1 -- 172.21.15.110:0/4141177074 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6b4f19230 con 0x55e6b4f1d840 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.592+0000 7fc889d9b640 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2e8a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc884009ca0 tx=0x7fc88402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59697009ccd0e334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.592+0000 7fc873fff640 1 -- 172.21.15.110:0/4141177074 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88405a070 con 0x55e6b4f1d840 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.592+0000 7fc873fff640 1 -- 172.21.15.110:0/4141177074 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc884036040 con 0x55e6b4f1d840 2024-04-25T00:14:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.592+0000 7fc873fff640 1 -- 172.21.15.110:0/4141177074 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884031e40 con 0x55e6b4f1d840 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.592+0000 7fc893795740 1 -- 172.21.15.110:0/4141177074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 msgr2=0x55e6b4f2e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.592+0000 7fc893795740 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2e8a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc884009ca0 tx=0x7fc88402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 -- 172.21.15.110:0/4141177074 shutdown_connections 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f229b0 0x55e6b4f26980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2e8a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 --2- 172.21.15.110:0/4141177074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f1bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 -- 172.21.15.110:0/4141177074 >> 172.21.15.110:0/4141177074 conn(0x55e6b4f184c0 msgr2=0x55e6b4f188a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 -- 172.21.15.110:0/4141177074 shutdown_connections 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 -- 172.21.15.110:0/4141177074 wait complete. 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 -- start start 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 Processor -- start 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f2bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1b1aa0 con 0x55e6b4f229b0 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1b2f10 con 0x55e6b4f1d840 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1b2170 con 0x55e6b4f1ce50 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.593+0000 7fc893795740 10 client.0 _ll_register_callbacks cb 0x7fc87c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:37538/0 (socket says 172.21.15.110:37538) 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc889d9b640 1 -- 172.21.15.110:0/2671541961 learned_addr learned my addr 172.21.15.110:0/2671541961 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc88a59c640 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f2bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc889d9b640 1 -- 172.21.15.110:0/2671541961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f1ce50 msgr2=0x55e6b4f2bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc889d9b640 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f2bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc889d9b640 1 -- 172.21.15.110:0/2671541961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f229b0 msgr2=0x55e6b4f2d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc889d9b640 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc889d9b640 1 -- 172.21.15.110:0/2671541961 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6b4f2d780 con 0x55e6b4f1d840 2024-04-25T00:14:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc88ad9d640 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc889d9b640 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2c180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc884006010 tx=0x7fc884031c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67a59ba434859bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc871ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc871ffb640 1 -- 172.21.15.110:0/2671541961 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88405a070 con 0x55e6b4f1d840 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc871ffb640 1 -- 172.21.15.110:0/2671541961 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc884003d80 con 0x55e6b4f1d840 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.594+0000 7fc893795740 1 -- 172.21.15.110:0/2671541961 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc884036040 con 0x55e6b4f1d840 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc871ffb640 1 -- 172.21.15.110:0/2671541961 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc884054410 con 0x55e6b4f1d840 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc893795740 10 client.25799 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc893795740 1 -- 172.21.15.110:0/2671541961 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6b4f2eac0 con 0x55e6b4f1d840 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc871ffb640 1 -- 172.21.15.110:0/2671541961 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc884030840 con 0x55e6b4f1d840 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc871ffb640 1 -- 172.21.15.110:0/2671541961 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc8840040d0 con 0x55e6b4f1d840 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc893795740 10 client.25799 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc893795740 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc893795740 20 client.25799 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc893795740 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e6b4f33330 0x55e6b4f9c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc893795740 4 client.25799 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:14:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc893795740 1 -- 172.21.15.110:0/2671541961 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc884036040 con 0x55e6b4f33330 2024-04-25T00:14:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.595+0000 7fc88a59c640 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e6b4f33330 0x55e6b4f9c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.596+0000 7fc88a59c640 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e6b4f33330 0x55e6b4f9c000 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb2e7b1364a43c2 server_cookie=41bb9da801e9d496 in_seq=0 out_seq=0 2024-04-25T00:14:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.596+0000 7fc871ffb640 10 client.25799 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.608+0000 7fc871ffb640 1 -- 172.21.15.110:0/2671541961 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc884036040 con 0x55e6b4f33330 2024-04-25T00:14:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.608+0000 7fc871ffb640 10 client.25799 handle_command_reply: tid=0 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.608+0000 7fc893795740 1 client.25799 shutdown 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 -- 172.21.15.110:0/2671541961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 msgr2=0x55e6b4f2c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2c180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc884006010 tx=0x7fc884031c30 comp rx=0 tx=0).stop 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 -- 172.21.15.110:0/2671541961 shutdown_connections 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55e6b4f33330 0x55e6b4f9c000 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6b4f229b0 0x55e6b4f2d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b4f1d840 0x55e6b4f2c180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 --2- 172.21.15.110:0/2671541961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e6b4f1ce50 0x55e6b4f2bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc871ffb640 0 client.25799 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 -- 172.21.15.110:0/2671541961 >> 172.21.15.110:0/2671541961 conn(0x55e6b4f184c0 msgr2=0x55e6b4f1bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 -- 172.21.15.110:0/2671541961 shutdown_connections 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 1 -- 172.21.15.110:0/2671541961 wait complete. 2024-04-25T00:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.609+0000 7fc893795740 20 client.25799 trim_cache size 0 max 16384 2024-04-25T00:14:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:14:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:14:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:14:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:14:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 -- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8740a2cb0 msgr2=0x7fc8740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8740a2cb0 0x7fc8740a5170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc88c1a37b0 tx=0x7fc884000f40 comp rx=0 tx=0).stop 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 -- 172.21.15.110:0/2654851743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c10ffc0 msgr2=0x7fc88c1a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c10ffc0 0x7fc88c1a9850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc88000e9b0 tx=0x7fc88000ee70 comp rx=0 tx=0).stop 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 -- 172.21.15.110:0/2654851743 shutdown_connections 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc8740a2cb0 0x7fc8740a5170 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c10ffc0 0x7fc88c1a9850 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc88c1090a0 0x7fc88c1a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 --2- 172.21.15.110:0/2654851743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc88c108740 0x7fc88c1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 -- 172.21.15.110:0/2654851743 >> 172.21.15.110:0/2654851743 conn(0x7fc88c06e7b0 msgr2=0x7fc88c0703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 -- 172.21.15.110:0/2654851743 shutdown_connections 2024-04-25T00:14:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:13.610+0000 7fc8909af640 1 -- 172.21.15.110:0/2654851743 wait complete. 2024-04-25T00:14:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:13 smithi173 ceph-mon[32402]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:14:14.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:13 smithi188 ceph-mon[33285]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:14:14.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:13 smithi110 ceph-mon[29187]: pgmap v716: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:14:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:15 smithi173 ceph-mon[32402]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 MiB/s wr, 935 op/s 2024-04-25T00:14:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:15 smithi188 ceph-mon[33285]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 MiB/s wr, 935 op/s 2024-04-25T00:14:16.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:15 smithi110 ceph-mon[29187]: pgmap v717: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 MiB/s wr, 935 op/s 2024-04-25T00:14:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 670.0 s, 850.2 tps, lat 117.365 ms stddev 21.349 2024-04-25T00:14:17.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:16 smithi188 ceph-mon[33285]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 852 op/s 2024-04-25T00:14:17.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:16 smithi110 ceph-mon[29187]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 852 op/s 2024-04-25T00:14:17.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:16 smithi173 ceph-mon[32402]: pgmap v718: 97 pgs: 97 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 852 op/s 2024-04-25T00:14:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:19 smithi173 ceph-mon[32402]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.5 MiB/s wr, 924 op/s 2024-04-25T00:14:20.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:19 smithi188 ceph-mon[33285]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.5 MiB/s wr, 924 op/s 2024-04-25T00:14:20.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:19 smithi110 ceph-mon[29187]: pgmap v719: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.5 MiB/s wr, 924 op/s 2024-04-25T00:14:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 675.0 s, 835.0 tps, lat 119.846 ms stddev 20.696 2024-04-25T00:14:22.088 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:21 smithi110 ceph-mon[29187]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-04-25T00:14:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:21 smithi173 ceph-mon[32402]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-04-25T00:14:22.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:21 smithi188 ceph-mon[33285]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 844 op/s 2024-04-25T00:14:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:23 smithi173 ceph-mon[32402]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:14:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:23 smithi188 ceph-mon[33285]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:14:24.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:23 smithi110 ceph-mon[29187]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-04-25T00:14:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:25 smithi173 ceph-mon[32402]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 901 op/s 2024-04-25T00:14:26.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:25 smithi188 ceph-mon[33285]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 901 op/s 2024-04-25T00:14:26.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:25 smithi110 ceph-mon[29187]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 901 op/s 2024-04-25T00:14:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 680.0 s, 809.8 tps, lat 123.524 ms stddev 28.563 2024-04-25T00:14:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.740+0000 7f979069e640 1 -- 172.21.15.110:0/1890581092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 msgr2=0x7f97881123b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.740+0000 7f979069e640 1 --2- 172.21.15.110:0/1890581092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 0x7f97881123b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9780009ca0 tx=0x7f978002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.741+0000 7f979069e640 1 -- 172.21.15.110:0/1890581092 shutdown_connections 2024-04-25T00:14:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.741+0000 7f979069e640 1 --2- 172.21.15.110:0/1890581092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 0x7f97881123b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.741+0000 7f979069e640 1 --2- 172.21.15.110:0/1890581092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97881090a0 0x7f978810fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.741+0000 7f979069e640 1 --2- 172.21.15.110:0/1890581092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9788108740 0x7f9788108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.741+0000 7f979069e640 1 -- 172.21.15.110:0/1890581092 >> 172.21.15.110:0/1890581092 conn(0x7f978806e7b0 msgr2=0x7f9788070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.743 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.741+0000 7f979069e640 1 -- 172.21.15.110:0/1890581092 shutdown_connections 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.741+0000 7f979069e640 1 -- 172.21.15.110:0/1890581092 wait complete. 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f979069e640 1 Processor -- start 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f979069e640 1 -- start start 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f979069e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9788108740 0x7f978819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f979069e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97881090a0 0x7f97881a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f979069e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 0x7f978819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f979069e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9788115b00 con 0x7f97881090a0 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f979069e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9788115980 con 0x7f9788108740 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f979069e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9788115c80 con 0x7f978810ffc0 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f978dc39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97881090a0 0x7f97881a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f978dc39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97881090a0 0x7f97881a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38426/0 (socket says 172.21.15.110:38426) 2024-04-25T00:14:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.742+0000 7f978dc39640 1 -- 172.21.15.110:0/3485077826 learned_addr learned my addr 172.21.15.110:0/3485077826 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978e43a640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9788108740 0x7f978819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978ec3b640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 0x7f978819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978dc39640 1 -- 172.21.15.110:0/3485077826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 msgr2=0x7f978819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978dc39640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 0x7f978819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978dc39640 1 -- 172.21.15.110:0/3485077826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9788108740 msgr2=0x7f978819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978dc39640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9788108740 0x7f978819deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978dc39640 1 -- 172.21.15.110:0/3485077826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97881b6b10 con 0x7f97881090a0 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978ec3b640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 0x7f978819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.743+0000 7f978dc39640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97881090a0 0x7f97881a3450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f977c00d950 tx=0x7f977c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.744+0000 7f977b7fe640 1 -- 172.21.15.110:0/3485077826 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c017070 con 0x7f97881090a0 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.744+0000 7f977b7fe640 1 -- 172.21.15.110:0/3485077826 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f977c009c30 con 0x7f97881090a0 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.744+0000 7f977b7fe640 1 -- 172.21.15.110:0/3485077826 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c00f620 con 0x7f97881090a0 2024-04-25T00:14:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.744+0000 7f979069e640 1 -- 172.21.15.110:0/3485077826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97881b6e00 con 0x7f97881090a0 2024-04-25T00:14:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.744+0000 7f979069e640 1 -- 172.21.15.110:0/3485077826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97881b7310 con 0x7f97881090a0 2024-04-25T00:14:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.747+0000 7f97933af740 1 Processor -- start 2024-04-25T00:14:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f97933af740 1 -- start start 2024-04-25T00:14:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f977b7fe640 1 -- 172.21.15.110:0/3485077826 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f977c0109a0 con 0x7f97881090a0 2024-04-25T00:14:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f97933af740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f31de50 0x555d9f31e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f97933af740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31e840 0x555d9f323340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f97933af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f3239b0 0x555d9f325e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f97933af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555d9f281cb0 con 0x555d9f3239b0 2024-04-25T00:14:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f97933af740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555d9f22a5b0 con 0x555d9f31de50 2024-04-25T00:14:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f97933af740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555d9f27dc70 con 0x555d9f31e840 2024-04-25T00:14:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f978e43a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31e840 0x555d9f323340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.748+0000 7f978e43a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31e840 0x555d9f323340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:53162/0 (socket says 172.21.15.110:53162) 2024-04-25T00:14:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978e43a640 1 -- 172.21.15.110:0/254424955 learned_addr learned my addr 172.21.15.110:0/254424955 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978dc39640 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f3239b0 0x555d9f325e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978ec3b640 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f31de50 0x555d9f31e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978e43a640 1 -- 172.21.15.110:0/254424955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f31de50 msgr2=0x555d9f31e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978e43a640 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f31de50 0x555d9f31e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978e43a640 1 -- 172.21.15.110:0/254424955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f3239b0 msgr2=0x555d9f325e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978e43a640 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f3239b0 0x555d9f325e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978e43a640 1 -- 172.21.15.110:0/254424955 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9ef96210 con 0x555d9f31e840 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.749+0000 7f978e43a640 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31e840 0x555d9f323340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f978400b820 tx=0x7f978400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f6e2c203689560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.750+0000 7f9778ff9640 1 -- 172.21.15.110:0/254424955 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978400d950 con 0x555d9f31e840 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.750+0000 7f977b7fe640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97740a2db0 0x7f97740a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.750+0000 7f9778ff9640 1 -- 172.21.15.110:0/254424955 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9784019070 con 0x555d9f31e840 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.750+0000 7f977b7fe640 1 -- 172.21.15.110:0/3485077826 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f977c0cecb0 con 0x7f97881090a0 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.750+0000 7f978ec3b640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97740a2db0 0x7f97740a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f978ec3b640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97740a2db0 0x7f97740a5270 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9780031040 tx=0x7f9780005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f97933af740 1 -- 172.21.15.110:0/254424955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31e840 msgr2=0x555d9f323340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f9778ff9640 1 -- 172.21.15.110:0/254424955 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840123e0 con 0x555d9f31e840 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f97933af740 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31e840 0x555d9f323340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f978400b820 tx=0x7f978400bce0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f97933af740 1 -- 172.21.15.110:0/254424955 shutdown_connections 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f97933af740 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f3239b0 0x555d9f325e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f97933af740 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31e840 0x555d9f323340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f97933af740 1 --2- 172.21.15.110:0/254424955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f31de50 0x555d9f31e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.751+0000 7f97933af740 1 -- 172.21.15.110:0/254424955 >> 172.21.15.110:0/254424955 conn(0x555d9f3194c0 msgr2=0x555d9f3198c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.752+0000 7f97933af740 1 -- 172.21.15.110:0/254424955 shutdown_connections 2024-04-25T00:14:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.752+0000 7f97933af740 1 -- 172.21.15.110:0/254424955 wait complete. 2024-04-25T00:14:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.752+0000 7f97933af740 1 -- start start 2024-04-25T00:14:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.753+0000 7f97933af740 1 Processor -- start 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.753+0000 7f97933af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f31de50 0x555d9f328090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.753+0000 7f97933af740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f3285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.753+0000 7f97933af740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f32d600 0x555d9f32f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.753+0000 7f97933af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555d9f326a00 con 0x555d9f31de50 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.753+0000 7f97933af740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555d9f23db50 con 0x555d9f3239b0 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.753+0000 7f97933af740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555d9f326b80 con 0x555d9f32d600 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.753+0000 7f97933af740 10 client.0 _ll_register_callbacks cb 0x555d9f278ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.754+0000 7f978dc39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f3285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.754+0000 7f978dc39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f3285d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:50202/0 (socket says 172.21.15.110:50202) 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.754+0000 7f978dc39640 1 -- 172.21.15.110:0/3309655874 learned_addr learned my addr 172.21.15.110:0/3309655874 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.754+0000 7f978ec3b640 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f32d600 0x555d9f32f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.755+0000 7f978e43a640 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f31de50 0x555d9f328090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.755+0000 7f978dc39640 1 -- 172.21.15.110:0/3309655874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f32d600 msgr2=0x555d9f32f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.755+0000 7f978dc39640 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f32d600 0x555d9f32f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.755+0000 7f978dc39640 1 -- 172.21.15.110:0/3309655874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f31de50 msgr2=0x555d9f328090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.755+0000 7f978dc39640 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f31de50 0x555d9f328090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.755+0000 7f978dc39640 1 -- 172.21.15.110:0/3309655874 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d9f062620 con 0x555d9f3239b0 2024-04-25T00:14:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.756+0000 7f978dc39640 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f3285d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f977c097eb0 tx=0x7f977c09b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b64b83af59dc46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.756+0000 7f9771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:14:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.756+0000 7f9771ffb640 1 -- 172.21.15.110:0/3309655874 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c020070 con 0x555d9f3239b0 2024-04-25T00:14:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.756+0000 7f9771ffb640 1 -- 172.21.15.110:0/3309655874 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f977c097b50 con 0x555d9f3239b0 2024-04-25T00:14:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.756+0000 7f97933af740 1 -- 172.21.15.110:0/3309655874 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9784019070 con 0x555d9f3239b0 2024-04-25T00:14:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.757+0000 7f9771ffb640 1 -- 172.21.15.110:0/3309655874 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f977c09b4e0 con 0x555d9f3239b0 2024-04-25T00:14:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.757+0000 7f97933af740 10 client.25811 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.757+0000 7f97933af740 1 -- 172.21.15.110:0/3309655874 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d9f332e20 con 0x555d9f3239b0 2024-04-25T00:14:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.757+0000 7f9771ffb640 1 -- 172.21.15.110:0/3309655874 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f977c09f890 con 0x555d9f3239b0 2024-04-25T00:14:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.757+0000 7f9771ffb640 1 -- 172.21.15.110:0/3309655874 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f977c097230 con 0x555d9f3239b0 2024-04-25T00:14:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.757+0000 7f97933af740 10 client.25811 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.757+0000 7f97933af740 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.757+0000 7f97933af740 20 client.25811 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.758+0000 7f97933af740 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555d9f3344a0 0x555d9f336890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.758+0000 7f97933af740 4 client.25811 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.758+0000 7f97933af740 1 -- 172.21.15.110:0/3309655874 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9784019070 con 0x555d9f3344a0 2024-04-25T00:14:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.758+0000 7f978e43a640 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555d9f3344a0 0x555d9f336890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.758+0000 7f9771ffb640 10 client.25811 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.758+0000 7f978e43a640 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555d9f3344a0 0x555d9f336890 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1edbecbcf5b786f0 server_cookie=6c27e655ae680c66 in_seq=0 out_seq=0 2024-04-25T00:14:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.759+0000 7f9771ffb640 1 -- 172.21.15.110:0/3309655874 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f9784019070 con 0x555d9f3344a0 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.759+0000 7f9771ffb640 10 client.25811 handle_command_reply: tid=0 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.759+0000 7f97933af740 1 client.25811 shutdown 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 -- 172.21.15.110:0/3309655874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 msgr2=0x555d9f3285d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f3285d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f977c097eb0 tx=0x7f977c09b370 comp rx=0 tx=0).stop 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 -- 172.21.15.110:0/3309655874 shutdown_connections 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555d9f3344a0 0x555d9f336890 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f9771ffb640 0 client.25811 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f32d600 0x555d9f32f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f3285d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 --2- 172.21.15.110:0/3309655874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f31de50 0x555d9f328090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 -- 172.21.15.110:0/3309655874 >> 172.21.15.110:0/3309655874 conn(0x555d9f3194c0 msgr2=0x555d9f3185c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.760+0000 7f97933af740 1 -- 172.21.15.110:0/3309655874 shutdown_connections 2024-04-25T00:14:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.761+0000 7f97933af740 1 -- 172.21.15.110:0/3309655874 wait complete. 2024-04-25T00:14:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.761+0000 7f97933af740 20 client.25811 trim_cache size 0 max 16384 2024-04-25T00:14:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.785+0000 7f97933af740 1 Processor -- start 2024-04-25T00:14:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.785+0000 7f97933af740 1 -- start start 2024-04-25T00:14:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f97933af740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f31f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f97933af740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f31ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f97933af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f328090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978e43a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f31f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978e43a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f31f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:53178/0 (socket says 172.21.15.110:53178) 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f97933af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97881c2970 con 0x555d9f32d600 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f97933af740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f97881c0240 con 0x555d9f3239b0 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f97933af740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97881be090 con 0x555d9f31de50 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978dc39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f31ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978e43a640 1 -- 172.21.15.110:0/540441370 learned_addr learned my addr 172.21.15.110:0/540441370 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978ec3b640 1 --2- 172.21.15.110:0/540441370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f328090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978dc39640 1 -- 172.21.15.110:0/540441370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 msgr2=0x555d9f31f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978dc39640 1 --2- 172.21.15.110:0/540441370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f31f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978dc39640 1 -- 172.21.15.110:0/540441370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 msgr2=0x555d9f328090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978dc39640 1 --2- 172.21.15.110:0/540441370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f328090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.786+0000 7f978dc39640 1 -- 172.21.15.110:0/540441370 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9f336c00 con 0x555d9f3239b0 2024-04-25T00:14:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.787+0000 7f978dc39640 1 --2- 172.21.15.110:0/540441370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f31ff00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f977c0a3c20 tx=0x7f977c0999f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=263fdaba3b63124c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.787+0000 7f9773fff640 1 -- 172.21.15.110:0/540441370 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c020030 con 0x555d9f3239b0 2024-04-25T00:14:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.787+0000 7f9773fff640 1 -- 172.21.15.110:0/540441370 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f977c0b8020 con 0x555d9f3239b0 2024-04-25T00:14:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.787+0000 7f9773fff640 1 -- 172.21.15.110:0/540441370 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c097b50 con 0x555d9f3239b0 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 -- 172.21.15.110:0/540441370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 msgr2=0x555d9f31ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 --2- 172.21.15.110:0/540441370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f31ff00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f977c0a3c20 tx=0x7f977c0999f0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 -- 172.21.15.110:0/540441370 shutdown_connections 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 --2- 172.21.15.110:0/540441370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f328090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 --2- 172.21.15.110:0/540441370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f31ff00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 --2- 172.21.15.110:0/540441370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f31f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 -- 172.21.15.110:0/540441370 >> 172.21.15.110:0/540441370 conn(0x555d9f3194c0 msgr2=0x555d9f3198a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 -- 172.21.15.110:0/540441370 shutdown_connections 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 -- 172.21.15.110:0/540441370 wait complete. 2024-04-25T00:14:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 -- start start 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.788+0000 7f97933af740 1 Processor -- start 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f97933af740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f3453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f97933af740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f345900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f97933af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f346910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f97933af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97881be700 con 0x555d9f32d600 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f97933af740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f97881bfb70 con 0x555d9f3239b0 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f97933af740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97881bedd0 con 0x555d9f31de50 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f97933af740 10 client.0 _ll_register_callbacks cb 0x7f977c097780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978dc39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f345900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978e43a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f3453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978dc39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f345900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:50212/0 (socket says 172.21.15.110:50212) 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978dc39640 1 -- 172.21.15.110:0/1034109090 learned_addr learned my addr 172.21.15.110:0/1034109090 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978ec3b640 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f346910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978dc39640 1 -- 172.21.15.110:0/1034109090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 msgr2=0x555d9f3453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978dc39640 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f3453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978dc39640 1 -- 172.21.15.110:0/1034109090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 msgr2=0x555d9f346910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.789+0000 7f978dc39640 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f346910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.790+0000 7f978dc39640 1 -- 172.21.15.110:0/1034109090 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d9f346f00 con 0x555d9f3239b0 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.790+0000 7f978ec3b640 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f346910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.790+0000 7f978dc39640 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f345900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f977c0976f0 tx=0x7f977c010570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee82448b9af4d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.790+0000 7f9771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.790+0000 7f9771ffb640 1 -- 172.21.15.110:0/1034109090 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c080030 con 0x555d9f3239b0 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.791+0000 7f9771ffb640 1 -- 172.21.15.110:0/1034109090 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f977c0b9070 con 0x555d9f3239b0 2024-04-25T00:14:26.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.791+0000 7f97933af740 1 -- 172.21.15.110:0/1034109090 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f977c0b8020 con 0x555d9f3239b0 2024-04-25T00:14:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.791+0000 7f9771ffb640 1 -- 172.21.15.110:0/1034109090 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f977c020050 con 0x555d9f3239b0 2024-04-25T00:14:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.791+0000 7f97933af740 10 client.25817 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.791+0000 7f97933af740 1 -- 172.21.15.110:0/1034109090 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d9f346f00 con 0x555d9f3239b0 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.791+0000 7f9771ffb640 1 -- 172.21.15.110:0/1034109090 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f977c0b8020 con 0x555d9f3239b0 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.792+0000 7f9771ffb640 1 -- 172.21.15.110:0/1034109090 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f977c09a3a0 con 0x555d9f3239b0 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.792+0000 7f97933af740 10 client.25817 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.792+0000 7f97933af740 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.792+0000 7f97933af740 20 client.25817 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.792+0000 7f97933af740 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555d9f3344a0 0x555d9f39c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.792+0000 7f97933af740 4 client.25817 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.792+0000 7f97933af740 1 -- 172.21.15.110:0/1034109090 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d9f28f1f0 con 0x555d9f3344a0 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.792+0000 7f978e43a640 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555d9f3344a0 0x555d9f39c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.793+0000 7f978e43a640 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555d9f3344a0 0x555d9f39c9d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9dca40b01bbc79b server_cookie=a263155f24967e38 in_seq=0 out_seq=0 2024-04-25T00:14:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.793+0000 7f9771ffb640 10 client.25817 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f9771ffb640 1 -- 172.21.15.110:0/1034109090 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d9f28f1f0 con 0x555d9f3344a0 2024-04-25T00:14:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f9771ffb640 10 client.25817 handle_command_reply: tid=0 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 client.25817 shutdown 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 -- 172.21.15.110:0/1034109090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 msgr2=0x555d9f345900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f345900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f977c0976f0 tx=0x7f977c010570 comp rx=0 tx=0).stop 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 -- 172.21.15.110:0/1034109090 shutdown_connections 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x555d9f3344a0 0x555d9f39c9d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555d9f32d600 0x555d9f346910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f9771ffb640 0 client.25817 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d9f3239b0 0x555d9f345900 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 --2- 172.21.15.110:0/1034109090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555d9f31de50 0x555d9f3453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 -- 172.21.15.110:0/1034109090 >> 172.21.15.110:0/1034109090 conn(0x555d9f3194c0 msgr2=0x555d9f31f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 -- 172.21.15.110:0/1034109090 shutdown_connections 2024-04-25T00:14:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.817+0000 7f97933af740 1 -- 172.21.15.110:0/1034109090 wait complete. 2024-04-25T00:14:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.818+0000 7f97933af740 20 client.25817 trim_cache size 0 max 16384 2024-04-25T00:14:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:14:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:14:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:14:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:14:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.818+0000 7f979069e640 1 -- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97740a2db0 msgr2=0x7f97740a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.818+0000 7f979069e640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97740a2db0 0x7f97740a5270 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9780031040 tx=0x7f9780005f80 comp rx=0 tx=0).stop 2024-04-25T00:14:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.818+0000 7f979069e640 1 -- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97881090a0 msgr2=0x7f97881a3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.818+0000 7f979069e640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97881090a0 0x7f97881a3450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f977c00d950 tx=0x7f977c00de10 comp rx=0 tx=0).stop 2024-04-25T00:14:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.819+0000 7f979069e640 1 -- 172.21.15.110:0/3485077826 shutdown_connections 2024-04-25T00:14:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.819+0000 7f979069e640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f97740a2db0 0x7f97740a5270 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.819+0000 7f979069e640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978810ffc0 0x7f978819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.819+0000 7f979069e640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97881090a0 0x7f97881a3450 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.819+0000 7f979069e640 1 --2- 172.21.15.110:0/3485077826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9788108740 0x7f978819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.819+0000 7f979069e640 1 -- 172.21.15.110:0/3485077826 >> 172.21.15.110:0/3485077826 conn(0x7f978806e7b0 msgr2=0x7f9788109850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.819+0000 7f979069e640 1 -- 172.21.15.110:0/3485077826 shutdown_connections 2024-04-25T00:14:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:26.819+0000 7f979069e640 1 -- 172.21.15.110:0/3485077826 wait complete. 2024-04-25T00:14:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:27 smithi173 ceph-mon[32402]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 815 op/s 2024-04-25T00:14:28.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:27 smithi188 ceph-mon[33285]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 815 op/s 2024-04-25T00:14:28.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:27 smithi110 ceph-mon[29187]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 815 op/s 2024-04-25T00:14:30.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:29 smithi173 ceph-mon[32402]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-04-25T00:14:30.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:29 smithi188 ceph-mon[33285]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-04-25T00:14:30.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:29 smithi110 ceph-mon[29187]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-04-25T00:14:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 685.0 s, 803.0 tps, lat 124.606 ms stddev 37.713 2024-04-25T00:14:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:31 smithi173 ceph-mon[32402]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-04-25T00:14:32.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:31 smithi188 ceph-mon[33285]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-04-25T00:14:32.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:31 smithi110 ceph-mon[29187]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-04-25T00:14:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:33 smithi173 ceph-mon[32402]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.8 MiB/s wr, 997 op/s 2024-04-25T00:14:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:33 smithi188 ceph-mon[33285]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.8 MiB/s wr, 997 op/s 2024-04-25T00:14:34.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:33 smithi110 ceph-mon[29187]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.8 MiB/s wr, 997 op/s 2024-04-25T00:14:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:35 smithi173 ceph-mon[32402]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 790 op/s 2024-04-25T00:14:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:35 smithi188 ceph-mon[33285]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 790 op/s 2024-04-25T00:14:36.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:35 smithi110 ceph-mon[29187]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 790 op/s 2024-04-25T00:14:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 690.0 s, 847.0 tps, lat 118.020 ms stddev 20.763 2024-04-25T00:14:38.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:37 smithi173 ceph-mon[32402]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-04-25T00:14:38.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:37 smithi188 ceph-mon[33285]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-04-25T00:14:38.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:37 smithi110 ceph-mon[29187]: pgmap v728: 97 pgs: 97 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 752 op/s 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 -- 172.21.15.110:0/2709618480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8102250 msgr2=0x7f33f8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2709618480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8102250 0x7f33f8108ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009900 tx=0x7f33f002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 -- 172.21.15.110:0/2709618480 shutdown_connections 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2709618480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8102250 0x7f33f8108ae0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2709618480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f8101890 0x7f33f8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2709618480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f81006a0 0x7f33f8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 -- 172.21.15.110:0/2709618480 >> 172.21.15.110:0/2709618480 conn(0x7f33f80fbc10 msgr2=0x7f33f80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 -- 172.21.15.110:0/2709618480 shutdown_connections 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 -- 172.21.15.110:0/2709618480 wait complete. 2024-04-25T00:14:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.933+0000 7f33fd71b640 1 Processor -- start 2024-04-25T00:14:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33fd71b640 1 -- start start 2024-04-25T00:14:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33fd71b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f81006a0 0x7f33f81956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33fd71b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8101890 0x7f33f8195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33fd71b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f8102250 0x7f33f819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33fd71b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33f810b0d0 con 0x7f33f8101890 2024-04-25T00:14:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33fd71b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f33f810af50 con 0x7f33f8102250 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33fd71b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33f810b250 con 0x7f33f81006a0 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f81006a0 0x7f33f81956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33f77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f8102250 0x7f33f819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f81006a0 0x7f33f81956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49738/0 (socket says 172.21.15.110:49738) 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33f77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f8102250 0x7f33f819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:57862/0 (socket says 172.21.15.110:57862) 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33f6ffd640 1 -- 172.21.15.110:0/2779611902 learned_addr learned my addr 172.21.15.110:0/2779611902 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.934+0000 7f33f67fc640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8101890 0x7f33f8195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33f77fe640 1 -- 172.21.15.110:0/2779611902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f81006a0 msgr2=0x7f33f81956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33f77fe640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f81006a0 0x7f33f81956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33f77fe640 1 -- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8101890 msgr2=0x7f33f8195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33f77fe640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8101890 0x7f33f8195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33f77fe640 1 -- 172.21.15.110:0/2779611902 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f819d2f0 con 0x7f33f8102250 2024-04-25T00:14:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33f67fc640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8101890 0x7f33f8195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33f77fe640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f8102250 0x7f33f819cc80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005940 tx=0x7f33f00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33dffff640 1 -- 172.21.15.110:0/2779611902 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f003f030 con 0x7f33f8102250 2024-04-25T00:14:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33dffff640 1 -- 172.21.15.110:0/2779611902 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33f00318f0 con 0x7f33f8102250 2024-04-25T00:14:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33fd71b640 1 -- 172.21.15.110:0/2779611902 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f819d580 con 0x7f33f8102250 2024-04-25T00:14:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33dffff640 1 -- 172.21.15.110:0/2779611902 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0035070 con 0x7f33f8102250 2024-04-25T00:14:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.935+0000 7f33fd71b640 1 -- 172.21.15.110:0/2779611902 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f819da70 con 0x7f33f8102250 2024-04-25T00:14:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.938+0000 7f33dffff640 1 -- 172.21.15.110:0/2779611902 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f33f0042030 con 0x7f33f8102250 2024-04-25T00:14:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.938+0000 7f340042f740 1 Processor -- start 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f340042f740 1 -- start start 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f33dffff640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f33e00a2c30 0x7f33e00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f340042f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab22e50 0x55761ab23270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f340042f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab23840 0x55761ab28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f340042f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab2ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f340042f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55761aa86cb0 con 0x55761ab23840 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f340042f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55761aa2f5b0 con 0x55761ab22e50 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f340042f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55761aa82c70 con 0x55761ab289b0 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f33f67fc640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f33e00a2c30 0x7f33e00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f33dffff640 1 -- 172.21.15.110:0/2779611902 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f33f00ef410 con 0x7f33f8102250 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab22e50 0x55761ab23270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab22e50 0x55761ab23270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:57864/0 (socket says 172.21.15.110:57864) 2024-04-25T00:14:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f33f6ffd640 1 -- 172.21.15.110:0/1046441977 learned_addr learned my addr 172.21.15.110:0/1046441977 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f33f67fc640 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab23840 0x55761ab28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.939+0000 7f33f77fe640 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab2ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33f67fc640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f33e00a2c30 0x7f33e00a50f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f33e800bfd0 tx=0x7f33e8004410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33f6ffd640 1 -- 172.21.15.110:0/1046441977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 msgr2=0x55761ab2ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab2ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33f6ffd640 1 -- 172.21.15.110:0/1046441977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab23840 msgr2=0x55761ab28340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab23840 0x55761ab28340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33f6ffd640 1 -- 172.21.15.110:0/1046441977 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761a79b210 con 0x55761ab22e50 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33f67fc640 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab23840 0x55761ab28340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab22e50 0x55761ab23270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009d70 tx=0x7f33ec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e3d86c019e2713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33dd7fa640 1 -- 172.21.15.110:0/1046441977 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec004220 con 0x55761ab22e50 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33dd7fa640 1 -- 172.21.15.110:0/1046441977 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33ec0043c0 con 0x55761ab22e50 2024-04-25T00:14:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.940+0000 7f33dd7fa640 1 -- 172.21.15.110:0/1046441977 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec019860 con 0x55761ab22e50 2024-04-25T00:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 -- 172.21.15.110:0/1046441977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab22e50 msgr2=0x55761ab23270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab22e50 0x55761ab23270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009d70 tx=0x7f33ec011810 comp rx=0 tx=0).stop 2024-04-25T00:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 -- 172.21.15.110:0/1046441977 shutdown_connections 2024-04-25T00:14:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab2ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab23840 0x55761ab28340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 --2- 172.21.15.110:0/1046441977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab22e50 0x55761ab23270 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 -- 172.21.15.110:0/1046441977 >> 172.21.15.110:0/1046441977 conn(0x55761ab1e4c0 msgr2=0x55761ab1e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 -- 172.21.15.110:0/1046441977 shutdown_connections 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.941+0000 7f340042f740 1 -- 172.21.15.110:0/1046441977 wait complete. 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 1 -- start start 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 1 Processor -- start 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab2d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab23840 0x55761ab2d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab289b0 0x55761ab34770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55761ab2b960 con 0x55761ab22e50 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55761aa42b50 con 0x55761ab289b0 2024-04-25T00:14:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55761ab2bae0 con 0x55761ab23840 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f33f77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab289b0 0x55761ab34770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f340042f740 10 client.0 _ll_register_callbacks cb 0x55761aa7dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f33f77fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab289b0 0x55761ab34770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:57868/0 (socket says 172.21.15.110:57868) 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f33f77fe640 1 -- 172.21.15.110:0/883616380 learned_addr learned my addr 172.21.15.110:0/883616380 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f33f67fc640 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab23840 0x55761ab2d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.942+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab2d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33f77fe640 1 -- 172.21.15.110:0/883616380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab23840 msgr2=0x55761ab2d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33f77fe640 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab23840 0x55761ab2d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33f77fe640 1 -- 172.21.15.110:0/883616380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 msgr2=0x55761ab2d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33f77fe640 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab2d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33f77fe640 1 -- 172.21.15.110:0/883616380 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761a867620 con 0x55761ab289b0 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab2d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33f77fe640 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab289b0 0x55761ab34770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33f00ecf40 tx=0x7f33f00bb770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cb54c47927b85e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33daff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33daff5640 1 -- 172.21.15.110:0/883616380 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0049040 con 0x55761ab289b0 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33daff5640 1 -- 172.21.15.110:0/883616380 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f33f00ef0c0 con 0x55761ab289b0 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f340042f740 1 -- 172.21.15.110:0/883616380 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec0043c0 con 0x55761ab289b0 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33daff5640 1 -- 172.21.15.110:0/883616380 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f33f00c0050 con 0x55761ab289b0 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f340042f740 10 client.25829 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f340042f740 1 -- 172.21.15.110:0/883616380 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55761ab37ae0 con 0x55761ab289b0 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.943+0000 7f33daff5640 1 -- 172.21.15.110:0/883616380 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33ec0043c0 con 0x55761ab289b0 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f33daff5640 1 -- 172.21.15.110:0/883616380 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f33f00c4480 con 0x55761ab289b0 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f340042f740 10 client.25829 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f340042f740 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f340042f740 20 client.25829 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f340042f740 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55761ab39200 0x55761ab3b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f340042f740 4 client.25829 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f340042f740 1 -- 172.21.15.110:0/883616380 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55761aa941f0 con 0x55761ab39200 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55761ab39200 0x55761ab3b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f33daff5640 10 client.25829 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.944+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55761ab39200 0x55761ab3b670 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa05ff3ed75cd7cf server_cookie=8511575bd6b966fc in_seq=0 out_seq=0 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.945+0000 7f33daff5640 1 -- 172.21.15.110:0/883616380 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55761aa941f0 con 0x55761ab39200 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.945+0000 7f33daff5640 10 client.25829 handle_command_reply: tid=0 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.945+0000 7f340042f740 1 client.25829 shutdown 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.945+0000 7f340042f740 1 -- 172.21.15.110:0/883616380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab289b0 msgr2=0x55761ab34770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.945+0000 7f340042f740 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab289b0 0x55761ab34770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33f00ecf40 tx=0x7f33f00bb770 comp rx=0 tx=0).stop 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 1 -- 172.21.15.110:0/883616380 shutdown_connections 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55761ab39200 0x55761ab3b670 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab289b0 0x55761ab34770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f33daff5640 0 client.25829 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab23840 0x55761ab2d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 1 --2- 172.21.15.110:0/883616380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab2d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 1 -- 172.21.15.110:0/883616380 >> 172.21.15.110:0/883616380 conn(0x55761ab1e4c0 msgr2=0x55761ab1d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 1 -- 172.21.15.110:0/883616380 shutdown_connections 2024-04-25T00:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 1 -- 172.21.15.110:0/883616380 wait complete. 2024-04-25T00:14:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.946+0000 7f340042f740 20 client.25829 trim_cache size 0 max 16384 2024-04-25T00:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.970+0000 7f340042f740 1 Processor -- start 2024-04-25T00:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f340042f740 1 -- start start 2024-04-25T00:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f340042f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f340042f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab34730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f340042f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f340042f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81a9080 con 0x55761ab22e50 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f340042f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81a6950 con 0x55761ab23840 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f340042f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81a47a0 con 0x55761ab289b0 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab34730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab242e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60124/0 (socket says 172.21.15.110:60124) 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 -- 172.21.15.110:0/2235327219 learned_addr learned my addr 172.21.15.110:0/2235327219 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f77fe640 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab21470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 -- 172.21.15.110:0/2235327219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 msgr2=0x55761ab21470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab21470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 -- 172.21.15.110:0/2235327219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 msgr2=0x55761ab34730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab34730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 -- 172.21.15.110:0/2235327219 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761ab1e060 con 0x55761ab22e50 2024-04-25T00:14:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f77fe640 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.971+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab242e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f33ec013010 tx=0x7f33ec009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606975f955dc301d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.972+0000 7f33dcff9640 1 -- 172.21.15.110:0/2235327219 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec030070 con 0x55761ab22e50 2024-04-25T00:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.972+0000 7f33dcff9640 1 -- 172.21.15.110:0/2235327219 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33ec004310 con 0x55761ab22e50 2024-04-25T00:14:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.972+0000 7f33dcff9640 1 -- 172.21.15.110:0/2235327219 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec003a90 con 0x55761ab22e50 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 -- 172.21.15.110:0/2235327219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 msgr2=0x55761ab242e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab242e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f33ec013010 tx=0x7f33ec009ce0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 -- 172.21.15.110:0/2235327219 shutdown_connections 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab34730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 --2- 172.21.15.110:0/2235327219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab242e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 -- 172.21.15.110:0/2235327219 >> 172.21.15.110:0/2235327219 conn(0x55761ab1e4c0 msgr2=0x55761ab2d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 -- 172.21.15.110:0/2235327219 shutdown_connections 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 -- 172.21.15.110:0/2235327219 wait complete. 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 -- start start 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.973+0000 7f340042f740 1 Processor -- start 2024-04-25T00:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f340042f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab31750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f340042f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab31c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f340042f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab9d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f340042f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81a4e10 con 0x55761ab22e50 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f340042f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81a6280 con 0x55761ab23840 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab31750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f340042f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33f81a54e0 con 0x55761ab289b0 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab31750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60130/0 (socket says 172.21.15.110:60130) 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f33f6ffd640 1 -- 172.21.15.110:0/2651251847 learned_addr learned my addr 172.21.15.110:0/2651251847 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f33f77fe640 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab9d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f340042f740 10 client.0 _ll_register_callbacks cb 0x7f33f00bbb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f33f67fc640 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab31c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.974+0000 7f33f67fc640 1 -- 172.21.15.110:0/2651251847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 msgr2=0x55761ab9d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33f67fc640 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab9d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33f67fc640 1 -- 172.21.15.110:0/2651251847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 msgr2=0x55761ab31750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33f67fc640 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab31750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33f67fc640 1 -- 172.21.15.110:0/2651251847 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761ab9dda0 con 0x55761ab23840 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33f77fe640 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab9d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33f67fc640 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab31c90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55761ab2e6d0 tx=0x7f33e8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69695f32e42e521d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33daff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33daff5640 1 -- 172.21.15.110:0/2651251847 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8004b20 con 0x55761ab23840 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33daff5640 1 -- 172.21.15.110:0/2651251847 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f33e8004cc0 con 0x55761ab23840 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f340042f740 1 -- 172.21.15.110:0/2651251847 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec004310 con 0x55761ab23840 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.975+0000 7f33daff5640 1 -- 172.21.15.110:0/2651251847 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f33e8038070 con 0x55761ab23840 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f340042f740 10 client.25835 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f340042f740 1 -- 172.21.15.110:0/2651251847 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55761ab9f150 con 0x55761ab23840 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f33daff5640 1 -- 172.21.15.110:0/2651251847 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33e80054d0 con 0x55761ab23840 2024-04-25T00:14:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f33daff5640 1 -- 172.21.15.110:0/2651251847 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f33e8025040 con 0x55761ab23840 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f340042f740 10 client.25835 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f340042f740 10 client.25835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f340042f740 20 client.25835 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f340042f740 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55761ab39200 0x55761aba1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f340042f740 4 client.25835 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f340042f740 1 -- 172.21.15.110:0/2651251847 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f33ec004310 con 0x55761ab39200 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.976+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55761ab39200 0x55761aba1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.977+0000 7f33daff5640 10 client.25835 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:39.978+0000 7f33f6ffd640 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55761ab39200 0x55761aba1b60 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbdd92ff304910d8 server_cookie=f4572d0abd4372f3 in_seq=0 out_seq=0 2024-04-25T00:14:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.005+0000 7f33daff5640 1 -- 172.21.15.110:0/2651251847 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f33ec004310 con 0x55761ab39200 2024-04-25T00:14:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.005+0000 7f33daff5640 10 client.25835 handle_command_reply: tid=0 2024-04-25T00:14:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.005+0000 7f340042f740 1 client.25835 shutdown 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 -- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 msgr2=0x55761ab31c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab31c90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55761ab2e6d0 tx=0x7f33e8004ab0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 -- 172.21.15.110:0/2651251847 shutdown_connections 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55761ab39200 0x55761aba1b60 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55761ab289b0 0x55761ab9d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55761ab23840 0x55761ab31c90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 --2- 172.21.15.110:0/2651251847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55761ab22e50 0x55761ab31750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 -- 172.21.15.110:0/2651251847 >> 172.21.15.110:0/2651251847 conn(0x55761ab1e4c0 msgr2=0x55761ab21470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f33daff5640 0 client.25835 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 -- 172.21.15.110:0/2651251847 shutdown_connections 2024-04-25T00:14:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 1 -- 172.21.15.110:0/2651251847 wait complete. 2024-04-25T00:14:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.006+0000 7f340042f740 20 client.25835 trim_cache size 0 max 16384 2024-04-25T00:14:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:14:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:14:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:14:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:14:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.012+0000 7f33fd71b640 1 -- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f33e00a2c30 msgr2=0x7f33e00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.012+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f33e00a2c30 0x7f33e00a50f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f33e800bfd0 tx=0x7f33e8004410 comp rx=0 tx=0).stop 2024-04-25T00:14:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.012+0000 7f33fd71b640 1 -- 172.21.15.110:0/2779611902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f8102250 msgr2=0x7f33f819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.012+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f8102250 0x7f33f819cc80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005940 tx=0x7f33f00042f0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.014+0000 7f33fd71b640 1 -- 172.21.15.110:0/2779611902 shutdown_connections 2024-04-25T00:14:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.014+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f33e00a2c30 0x7f33e00a50f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.014+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f8102250 0x7f33f819cc80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.014+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8101890 0x7f33f8195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.014+0000 7f33fd71b640 1 --2- 172.21.15.110:0/2779611902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f81006a0 0x7f33f81956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.014+0000 7f33fd71b640 1 -- 172.21.15.110:0/2779611902 >> 172.21.15.110:0/2779611902 conn(0x7f33f80fbc10 msgr2=0x7f33f8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.014+0000 7f33fd71b640 1 -- 172.21.15.110:0/2779611902 shutdown_connections 2024-04-25T00:14:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:40.014+0000 7f33fd71b640 1 -- 172.21.15.110:0/2779611902 wait complete. 2024-04-25T00:14:40.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:39 smithi188 ceph-mon[33285]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.9 MiB/s wr, 988 op/s 2024-04-25T00:14:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:39 smithi173 ceph-mon[32402]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.9 MiB/s wr, 988 op/s 2024-04-25T00:14:40.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:39 smithi110 ceph-mon[29187]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.9 MiB/s wr, 988 op/s 2024-04-25T00:14:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:40 smithi173 ceph-mon[32402]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-04-25T00:14:41.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:40 smithi188 ceph-mon[33285]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-04-25T00:14:41.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:40 smithi110 ceph-mon[29187]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-04-25T00:14:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 695.0 s, 844.8 tps, lat 118.326 ms stddev 22.371 2024-04-25T00:14:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:43 smithi173 ceph-mon[32402]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.1 MiB/s wr, 1.01k op/s 2024-04-25T00:14:44.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:43 smithi188 ceph-mon[33285]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.1 MiB/s wr, 1.01k op/s 2024-04-25T00:14:44.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:43 smithi110 ceph-mon[29187]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.1 MiB/s wr, 1.01k op/s 2024-04-25T00:14:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:45 smithi173 ceph-mon[32402]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 937 op/s 2024-04-25T00:14:46.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:45 smithi188 ceph-mon[33285]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 937 op/s 2024-04-25T00:14:46.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:45 smithi110 ceph-mon[29187]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 937 op/s 2024-04-25T00:14:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 700.0 s, 833.8 tps, lat 120.026 ms stddev 23.185 2024-04-25T00:14:48.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:47 smithi173 ceph-mon[32402]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 863 op/s 2024-04-25T00:14:48.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:47 smithi188 ceph-mon[33285]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 863 op/s 2024-04-25T00:14:48.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:47 smithi110 ceph-mon[29187]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 863 op/s 2024-04-25T00:14:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:49 smithi173 ceph-mon[32402]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.3 MiB/s wr, 1.02k op/s 2024-04-25T00:14:50.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:49 smithi188 ceph-mon[33285]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.3 MiB/s wr, 1.02k op/s 2024-04-25T00:14:50.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:49 smithi110 ceph-mon[29187]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.3 MiB/s wr, 1.02k op/s 2024-04-25T00:14:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 705.0 s, 881.0 tps, lat 113.718 ms stddev 18.492 2024-04-25T00:14:52.088 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:51 smithi110 ceph-mon[29187]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 850 op/s 2024-04-25T00:14:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:51 smithi173 ceph-mon[32402]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 850 op/s 2024-04-25T00:14:52.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:51 smithi188 ceph-mon[33285]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 850 op/s 2024-04-25T00:14:53.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:53.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:53.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:53.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:52 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 -- 172.21.15.110:0/4258707551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e60101380 msgr2=0x7f6e60101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 --2- 172.21.15.110:0/4258707551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e60101380 0x7f6e60101800 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009be0 tx=0x7f6e5802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 -- 172.21.15.110:0/4258707551 shutdown_connections 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 --2- 172.21.15.110:0/4258707551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e60101380 0x7f6e60101800 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 --2- 172.21.15.110:0/4258707551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e601009e0 0x7f6e60100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 --2- 172.21.15.110:0/4258707551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e60100080 0x7f6e601004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 -- 172.21.15.110:0/4258707551 >> 172.21.15.110:0/4258707551 conn(0x7f6e600fbc20 msgr2=0x7f6e600fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 -- 172.21.15.110:0/4258707551 shutdown_connections 2024-04-25T00:14:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.152+0000 7f6e64f90640 1 -- 172.21.15.110:0/4258707551 wait complete. 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e64f90640 1 Processor -- start 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e64f90640 1 -- start start 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e64f90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e60100080 0x7f6e60199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e64f90640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e601009e0 0x7f6e6019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e64f90640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e60101380 0x7f6e601a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e64f90640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010f4b0 con 0x7f6e601009e0 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e64f90640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010f330 con 0x7f6e60101380 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e64f90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010f630 con 0x7f6e60100080 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.153+0000 7f6e5e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e60100080 0x7f6e60199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e60100080 0x7f6e60199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:57640/0 (socket says 172.21.15.110:57640) 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2401081600 learned_addr learned my addr 172.21.15.110:0/2401081600 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e57fff640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e601009e0 0x7f6e6019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5ed9d640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e60101380 0x7f6e601a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2401081600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e60101380 msgr2=0x7f6e601a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e60101380 0x7f6e601a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e601009e0 msgr2=0x7f6e6019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e601009e0 0x7f6e6019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2401081600 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e601a16e0 con 0x7f6e60100080 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e60100080 0x7f6e60199ac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e9c0 tx=0x7f6e5000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e577fe640 1 -- 172.21.15.110:0/2401081600 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000cde0 con 0x7f6e60100080 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e577fe640 1 -- 172.21.15.110:0/2401081600 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e500035f0 con 0x7f6e60100080 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.154+0000 7f6e64f90640 1 -- 172.21.15.110:0/2401081600 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e601a19d0 con 0x7f6e60100080 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.155+0000 7f6e577fe640 1 -- 172.21.15.110:0/2401081600 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50010690 con 0x7f6e60100080 2024-04-25T00:14:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.155+0000 7f6e64f90640 1 -- 172.21.15.110:0/2401081600 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e601a1f70 con 0x7f6e60100080 2024-04-25T00:14:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.157+0000 7f6e577fe640 1 -- 172.21.15.110:0/2401081600 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6e500037a0 con 0x7f6e60100080 2024-04-25T00:14:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.158+0000 7f6e577fe640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6e4c0a2cb0 0x7f6e4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.158+0000 7f6e67d79740 1 Processor -- start 2024-04-25T00:14:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.158+0000 7f6e577fe640 1 -- 172.21.15.110:0/2401081600 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6e500cf260 con 0x7f6e60100080 2024-04-25T00:14:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.158+0000 7f6e57fff640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6e4c0a2cb0 0x7f6e4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.158+0000 7f6e67d79740 1 -- start start 2024-04-25T00:14:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e99a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e99f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e9a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e67d79740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557c1e8fdcb0 con 0x557c1e999e50 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e67d79740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557c1e8a65b0 con 0x557c1e99a840 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e67d79740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557c1e8f9c70 con 0x557c1e99f9b0 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e9a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e9a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:57652/0 (socket says 172.21.15.110:57652) 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 -- 172.21.15.110:0/1993586947 learned_addr learned my addr 172.21.15.110:0/1993586947 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e99a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e57fff640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6e4c0a2cb0 0x7f6e4c0a5170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6e6019afd0 tx=0x7f6e48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e57fff640 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e99f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 -- 172.21.15.110:0/1993586947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 msgr2=0x557c1e99f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e99f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 -- 172.21.15.110:0/1993586947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 msgr2=0x557c1e99a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e99a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 -- 172.21.15.110:0/1993586947 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c1e612210 con 0x557c1e99f9b0 2024-04-25T00:14:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.159+0000 7f6e5ed9d640 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e9a1e40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e5802acb0 tx=0x7f6e58005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b37f71e61a35e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.160+0000 7f6e54ff9640 1 -- 172.21.15.110:0/1993586947 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58041030 con 0x557c1e99f9b0 2024-04-25T00:14:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.160+0000 7f6e54ff9640 1 -- 172.21.15.110:0/1993586947 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e58037040 con 0x557c1e99f9b0 2024-04-25T00:14:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.160+0000 7f6e54ff9640 1 -- 172.21.15.110:0/1993586947 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58036040 con 0x557c1e99f9b0 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- 172.21.15.110:0/1993586947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 msgr2=0x557c1e9a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e9a1e40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e5802acb0 tx=0x7f6e58005ec0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- 172.21.15.110:0/1993586947 shutdown_connections 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e9a1e40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e99f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 --2- 172.21.15.110:0/1993586947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e99a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- 172.21.15.110:0/1993586947 >> 172.21.15.110:0/1993586947 conn(0x557c1e9954c0 msgr2=0x557c1e9958c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- 172.21.15.110:0/1993586947 shutdown_connections 2024-04-25T00:14:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- 172.21.15.110:0/1993586947 wait complete. 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- start start 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 Processor -- start 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e999e50 0x557c1e9a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99a840 0x557c1e9a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99f9b0 0x557c1e9ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557c1e9a2900 con 0x557c1e99f9b0 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557c1e8b9b50 con 0x557c1e999e50 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.161+0000 7f6e67d79740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557c1e9a2a80 con 0x557c1e99a840 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e999e50 0x557c1e9a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e67d79740 10 client.0 _ll_register_callbacks cb 0x557c1e8f4ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e999e50 0x557c1e9a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:35232/0 (socket says 172.21.15.110:35232) 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2410572687 learned_addr learned my addr 172.21.15.110:0/2410572687 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5ed9d640 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99f9b0 0x557c1e9ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e57fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99a840 0x557c1e9a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2410572687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99a840 msgr2=0x557c1e9a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99a840 0x557c1e9a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2410572687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99f9b0 msgr2=0x557c1e9ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99f9b0 0x557c1e9ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2410572687 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c1e6de620 con 0x557c1e999e50 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e57fff640 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99a840 0x557c1e9a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e999e50 0x557c1e9a4210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e500984c0 tx=0x7f6e50009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8078c345eed751e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e45ffb640 1 -- 172.21.15.110:0/2410572687 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5009b140 con 0x557c1e999e50 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e67d79740 1 -- 172.21.15.110:0/2410572687 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e58037040 con 0x557c1e999e50 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.162+0000 7f6e45ffb640 1 -- 172.21.15.110:0/2410572687 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6e500ba1c0 con 0x557c1e999e50 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e45ffb640 1 -- 172.21.15.110:0/2410572687 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e5009baf0 con 0x557c1e999e50 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e67d79740 10 client.25847 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e67d79740 1 -- 172.21.15.110:0/2410572687 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c1e9aebc0 con 0x557c1e999e50 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e45ffb640 1 -- 172.21.15.110:0/2410572687 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e58037040 con 0x557c1e999e50 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e45ffb640 1 -- 172.21.15.110:0/2410572687 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6e5009a9b0 con 0x557c1e999e50 2024-04-25T00:14:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e67d79740 10 client.25847 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e67d79740 10 client.25847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e67d79740 20 client.25847 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557c1e9b01e0 0x557c1e9b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e67d79740 4 client.25847 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.163+0000 7f6e67d79740 1 -- 172.21.15.110:0/2410572687 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c1e90b1f0 con 0x557c1e9b01e0 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.164+0000 7f6e57fff640 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557c1e9b01e0 0x557c1e9b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.164+0000 7f6e57fff640 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557c1e9b01e0 0x557c1e9b2630 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae895f44bfae736 server_cookie=8f97c95badec65c8 in_seq=0 out_seq=0 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.164+0000 7f6e45ffb640 10 client.25847 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e45ffb640 1 -- 172.21.15.110:0/2410572687 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c1e90b1f0 con 0x557c1e9b01e0 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e45ffb640 10 client.25847 handle_command_reply: tid=0 2024-04-25T00:14:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 client.25847 shutdown 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 -- 172.21.15.110:0/2410572687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e999e50 msgr2=0x557c1e9a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e999e50 0x557c1e9a4210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e500984c0 tx=0x7f6e50009510 comp rx=0 tx=0).stop 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 -- 172.21.15.110:0/2410572687 shutdown_connections 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557c1e9b01e0 0x557c1e9b2630 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99f9b0 0x557c1e9ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99a840 0x557c1e9a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2410572687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e999e50 0x557c1e9a4210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e67d79740 1 -- 172.21.15.110:0/2410572687 >> 172.21.15.110:0/2410572687 conn(0x557c1e9954c0 msgr2=0x557c1e9945c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.165+0000 7f6e45ffb640 0 client.25847 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.166+0000 7f6e67d79740 1 -- 172.21.15.110:0/2410572687 shutdown_connections 2024-04-25T00:14:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.166+0000 7f6e67d79740 1 -- 172.21.15.110:0/2410572687 wait complete. 2024-04-25T00:14:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.166+0000 7f6e67d79740 20 client.25847 trim_cache size 0 max 16384 2024-04-25T00:14:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e67d79740 1 Processor -- start 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e67d79740 1 -- start start 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e9a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e9ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e993d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e67d79740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601ad430 con 0x557c1e999e50 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e67d79740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601aad00 con 0x557c1e99a840 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e67d79740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a8b50 con 0x557c1e99f9b0 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e57fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e9ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e5e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e9a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e57fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e9ab720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:35238/0 (socket says 172.21.15.110:35238) 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e57fff640 1 -- 172.21.15.110:0/2698758760 learned_addr learned my addr 172.21.15.110:0/2698758760 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e5ed9d640 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e993d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2698758760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 msgr2=0x557c1e993d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e993d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2698758760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 msgr2=0x557c1e9ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e9ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e5e59c640 1 -- 172.21.15.110:0/2698758760 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c1e99f6d0 con 0x557c1e999e50 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.185+0000 7f6e57fff640 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e9ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.186+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e9a4530 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6e5009adf0 tx=0x7f6e5009a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07c06fadb6ee5b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.186+0000 7f6e47fff640 1 -- 172.21.15.110:0/2698758760 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e500a4760 con 0x557c1e999e50 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.186+0000 7f6e47fff640 1 -- 172.21.15.110:0/2698758760 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e5009fea0 con 0x557c1e999e50 2024-04-25T00:14:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.186+0000 7f6e47fff640 1 -- 172.21.15.110:0/2698758760 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50097840 con 0x557c1e999e50 2024-04-25T00:14:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.187+0000 7f6e67d79740 1 -- 172.21.15.110:0/2698758760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 msgr2=0x557c1e9a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.187+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e9a4530 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6e5009adf0 tx=0x7f6e5009a000 comp rx=0 tx=0).stop 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.187+0000 7f6e67d79740 1 -- 172.21.15.110:0/2698758760 shutdown_connections 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.187+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e99f9b0 0x557c1e993d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.187+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99a840 0x557c1e9ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.187+0000 7f6e67d79740 1 --2- 172.21.15.110:0/2698758760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e999e50 0x557c1e9a4530 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.187+0000 7f6e67d79740 1 -- 172.21.15.110:0/2698758760 >> 172.21.15.110:0/2698758760 conn(0x557c1e9954c0 msgr2=0x557c1e9a3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.187+0000 7f6e67d79740 1 -- 172.21.15.110:0/2698758760 shutdown_connections 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 -- 172.21.15.110:0/2698758760 wait complete. 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 -- start start 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 Processor -- start 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e999e50 0x557c1e9a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99a840 0x557c1e9a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99f9b0 0x557c1e9a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a91c0 con 0x557c1e99a840 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601aa630 con 0x557c1e99f9b0 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a9890 con 0x557c1e999e50 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.188+0000 7f6e67d79740 10 client.0 _ll_register_callbacks cb 0x7f6e5000c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99a840 0x557c1e9a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e5e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e999e50 0x557c1e9a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99a840 0x557c1e9a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38302/0 (socket says 172.21.15.110:38302) 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 -- 172.21.15.110:0/139178013 learned_addr learned my addr 172.21.15.110:0/139178013 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e5ed9d640 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99f9b0 0x557c1e9a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 -- 172.21.15.110:0/139178013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e999e50 msgr2=0x557c1e9a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e999e50 0x557c1e9a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 -- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99f9b0 msgr2=0x557c1e9a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99f9b0 0x557c1e9a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 -- 172.21.15.110:0/139178013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c1e9aa4b0 con 0x557c1e99a840 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e5ed9d640 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99f9b0 0x557c1e9a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e57fff640 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99a840 0x557c1e9a8eb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6e480118d0 tx=0x7f6e48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff0b9ea2abc95f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e45ffb640 1 -- 172.21.15.110:0/139178013 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4802a680 con 0x557c1e99a840 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e45ffb640 1 -- 172.21.15.110:0/139178013 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6e4802a820 con 0x557c1e99a840 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.189+0000 7f6e67d79740 1 -- 172.21.15.110:0/139178013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e5009fea0 con 0x557c1e99a840 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e45ffb640 1 -- 172.21.15.110:0/139178013 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e4801e8a0 con 0x557c1e99a840 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e67d79740 10 client.16257 fetch_fsmap learned FSMap version 32 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e67d79740 1 -- 172.21.15.110:0/139178013 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c1e9ab8c0 con 0x557c1e99a840 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e45ffb640 1 -- 172.21.15.110:0/139178013 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e5009fea0 con 0x557c1e99a840 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e45ffb640 1 -- 172.21.15.110:0/139178013 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6e48027050 con 0x557c1e99a840 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e67d79740 10 client.16257 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:14:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e67d79740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:14:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e67d79740 20 client.16257 populate_metadata read hostname 'smithi110' 2024-04-25T00:14:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e67d79740 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557c1ea16e90 0x557c1ea192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e67d79740 4 client.16257 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:14:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e67d79740 1 -- 172.21.15.110:0/139178013 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c1e905540 con 0x557c1ea16e90 2024-04-25T00:14:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.190+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557c1ea16e90 0x557c1ea192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.191+0000 7f6e5e59c640 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557c1ea16e90 0x557c1ea192b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f4e97324450e27 server_cookie=f25e6970df431869 in_seq=0 out_seq=0 2024-04-25T00:14:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.191+0000 7f6e45ffb640 10 client.16257 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:53.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e45ffb640 1 -- 172.21.15.110:0/139178013 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c1e905540 con 0x557c1ea16e90 2024-04-25T00:14:53.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e45ffb640 10 client.16257 handle_command_reply: tid=0 2024-04-25T00:14:53.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 client.16257 shutdown 2024-04-25T00:14:53.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 -- 172.21.15.110:0/139178013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99a840 msgr2=0x557c1e9a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.209 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99a840 0x557c1e9a8eb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6e480118d0 tx=0x7f6e48004730 comp rx=0 tx=0).stop 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 -- 172.21.15.110:0/139178013 shutdown_connections 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x557c1ea16e90 0x557c1ea192b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e45ffb640 0 client.16257 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557c1e99f9b0 0x557c1e9a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557c1e99a840 0x557c1e9a8eb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 --2- 172.21.15.110:0/139178013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c1e999e50 0x557c1e9a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 -- 172.21.15.110:0/139178013 >> 172.21.15.110:0/139178013 conn(0x557c1e9954c0 msgr2=0x557c1e9945c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 -- 172.21.15.110:0/139178013 shutdown_connections 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.208+0000 7f6e67d79740 1 -- 172.21.15.110:0/139178013 wait complete. 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e67d79740 20 client.16257 trim_cache size 0 max 16384 2024-04-25T00:14:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:14:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:14:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:14:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:14:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 -- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6e4c0a2cb0 msgr2=0x7f6e4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6e4c0a2cb0 0x7f6e4c0a5170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6e6019afd0 tx=0x7f6e48005f90 comp rx=0 tx=0).stop 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 -- 172.21.15.110:0/2401081600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e60100080 msgr2=0x7f6e60199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e60100080 0x7f6e60199ac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e9c0 tx=0x7f6e5000ee80 comp rx=0 tx=0).stop 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 -- 172.21.15.110:0/2401081600 shutdown_connections 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6e4c0a2cb0 0x7f6e4c0a5170 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e60101380 0x7f6e601a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e601009e0 0x7f6e6019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 --2- 172.21.15.110:0/2401081600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e60100080 0x7f6e60199ac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 -- 172.21.15.110:0/2401081600 >> 172.21.15.110:0/2401081600 conn(0x7f6e600fbc20 msgr2=0x7f6e600fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 -- 172.21.15.110:0/2401081600 shutdown_connections 2024-04-25T00:14:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:14:53.209+0000 7f6e64f90640 1 -- 172.21.15.110:0/2401081600 wait complete. 2024-04-25T00:14:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:52 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:14:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:52 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:14:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:53 smithi173 ceph-mon[32402]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.8 MiB/s wr, 982 op/s 2024-04-25T00:14:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:53 smithi188 ceph-mon[33285]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.8 MiB/s wr, 982 op/s 2024-04-25T00:14:54.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:53 smithi110 ceph-mon[29187]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.8 MiB/s wr, 982 op/s 2024-04-25T00:14:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:54 smithi173 ceph-mon[32402]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 867 op/s 2024-04-25T00:14:55.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:54 smithi188 ceph-mon[33285]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 867 op/s 2024-04-25T00:14:55.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:54 smithi110 ceph-mon[29187]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 867 op/s 2024-04-25T00:14:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 710.0 s, 862.4 tps, lat 115.719 ms stddev 17.775 2024-04-25T00:14:58.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:57 smithi173 ceph-mon[32402]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 MiB/s wr, 744 op/s 2024-04-25T00:14:58.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:57 smithi188 ceph-mon[33285]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 MiB/s wr, 744 op/s 2024-04-25T00:14:58.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:57 smithi110 ceph-mon[29187]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 MiB/s wr, 744 op/s 2024-04-25T00:15:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:14:59 smithi173 ceph-mon[32402]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.1 MiB/s wr, 990 op/s 2024-04-25T00:15:00.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:14:59 smithi188 ceph-mon[33285]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.1 MiB/s wr, 990 op/s 2024-04-25T00:15:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:14:59 smithi110 ceph-mon[29187]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.1 MiB/s wr, 990 op/s 2024-04-25T00:15:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 715.0 s, 828.8 tps, lat 120.538 ms stddev 24.454 2024-04-25T00:15:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:01 smithi173 ceph-mon[32402]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 853 op/s 2024-04-25T00:15:02.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:01 smithi188 ceph-mon[33285]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 853 op/s 2024-04-25T00:15:02.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:01 smithi110 ceph-mon[29187]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 853 op/s 2024-04-25T00:15:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:03 smithi173 ceph-mon[32402]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 986 op/s 2024-04-25T00:15:04.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:03 smithi188 ceph-mon[33285]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 986 op/s 2024-04-25T00:15:04.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:03 smithi110 ceph-mon[29187]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.9 MiB/s wr, 986 op/s 2024-04-25T00:15:06.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:05 smithi173 ceph-mon[32402]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 871 op/s 2024-04-25T00:15:06.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:05 smithi188 ceph-mon[33285]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 871 op/s 2024-04-25T00:15:06.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:05 smithi110 ceph-mon[29187]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 871 op/s 2024-04-25T00:15:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.340+0000 7fa7a0b30640 1 -- 172.21.15.110:0/3708014096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c1031c0 msgr2=0x7fa79c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.340+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/3708014096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c1031c0 0x7fa79c109a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa794009ca0 tx=0x7fa79402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.341+0000 7fa7a0b30640 1 -- 172.21.15.110:0/3708014096 shutdown_connections 2024-04-25T00:15:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.341+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/3708014096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c1031c0 0x7fa79c109a50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.341+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/3708014096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa79c102800 0x7fa79c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.341+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/3708014096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa79c101610 0x7fa79c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.341+0000 7fa7a0b30640 1 -- 172.21.15.110:0/3708014096 >> 172.21.15.110:0/3708014096 conn(0x7fa79c0fcba0 msgr2=0x7fa79c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.341+0000 7fa7a0b30640 1 -- 172.21.15.110:0/3708014096 shutdown_connections 2024-04-25T00:15:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.341+0000 7fa7a0b30640 1 -- 172.21.15.110:0/3708014096 wait complete. 2024-04-25T00:15:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa7a0b30640 1 Processor -- start 2024-04-25T00:15:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa7a0b30640 1 -- start start 2024-04-25T00:15:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa7a0b30640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c101610 0x7fa79c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa7a0b30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa79c102800 0x7fa79c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa7a0b30640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa79c1031c0 0x7fa79c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa79c102800 0x7fa79c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa79c102800 0x7fa79c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41760/0 (socket says 172.21.15.110:41760) 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa799d9b640 1 -- 172.21.15.110:0/955606402 learned_addr learned my addr 172.21.15.110:0/955606402 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c0740d0 con 0x7fa79c102800 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c073f50 con 0x7fa79c1031c0 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa79a59c640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c101610 0x7fa79c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c074250 con 0x7fa79c101610 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.342+0000 7fa79ad9d640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa79c1031c0 0x7fa79c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.343+0000 7fa799d9b640 1 -- 172.21.15.110:0/955606402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c101610 msgr2=0x7fa79c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.343+0000 7fa799d9b640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c101610 0x7fa79c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.343+0000 7fa799d9b640 1 -- 172.21.15.110:0/955606402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa79c1031c0 msgr2=0x7fa79c1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.343+0000 7fa799d9b640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa79c1031c0 0x7fa79c1a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.343+0000 7fa799d9b640 1 -- 172.21.15.110:0/955606402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c1a1670 con 0x7fa79c102800 2024-04-25T00:15:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.343+0000 7fa799d9b640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa79c102800 0x7fa79c199f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa79000ea60 tx=0x7fa79000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.343+0000 7fa78b7fe640 1 -- 172.21.15.110:0/955606402 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79000cd20 con 0x7fa79c102800 2024-04-25T00:15:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.344+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c1a1960 con 0x7fa79c102800 2024-04-25T00:15:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.344+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c1a1f00 con 0x7fa79c102800 2024-04-25T00:15:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.344+0000 7fa78b7fe640 1 -- 172.21.15.110:0/955606402 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7900035f0 con 0x7fa79c102800 2024-04-25T00:15:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.344+0000 7fa78b7fe640 1 -- 172.21.15.110:0/955606402 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790010690 con 0x7fa79c102800 2024-04-25T00:15:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.347+0000 7fa7a383e740 1 Processor -- start 2024-04-25T00:15:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.349+0000 7fa7a383e740 1 -- start start 2024-04-25T00:15:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.350+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2e7d0 0x55f105b2ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.350+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2f1c0 0x55f105b33cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.350+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b34330 0x55f105b367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.350+0000 7fa7a383e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f105a89cb0 con 0x55f105b34330 2024-04-25T00:15:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.350+0000 7fa7a383e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f105a325b0 con 0x55f105b2f1c0 2024-04-25T00:15:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.350+0000 7fa7a383e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f105a85c70 con 0x55f105b2e7d0 2024-04-25T00:15:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.351+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b34330 0x55f105b367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.351+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b34330 0x55f105b367c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41766/0 (socket says 172.21.15.110:41766) 2024-04-25T00:15:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.351+0000 7fa799d9b640 1 -- 172.21.15.110:0/1151867652 learned_addr learned my addr 172.21.15.110:0/1151867652 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.351+0000 7fa79ad9d640 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2e7d0 0x55f105b2ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.351+0000 7fa78b7fe640 1 -- 172.21.15.110:0/955606402 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa790003fa0 con 0x7fa79c102800 2024-04-25T00:15:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.351+0000 7fa79a59c640 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2f1c0 0x55f105b33cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.352+0000 7fa79ad9d640 1 -- 172.21.15.110:0/1151867652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2f1c0 msgr2=0x55f105b33cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.352+0000 7fa79ad9d640 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2f1c0 0x55f105b33cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.352+0000 7fa79ad9d640 1 -- 172.21.15.110:0/1151867652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b34330 msgr2=0x55f105b367c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.352+0000 7fa79ad9d640 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b34330 0x55f105b367c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.352+0000 7fa79ad9d640 1 -- 172.21.15.110:0/1151867652 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f10579e210 con 0x55f105b2e7d0 2024-04-25T00:15:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.352+0000 7fa79a59c640 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2f1c0 0x55f105b33cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.352+0000 7fa79ad9d640 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2e7d0 0x55f105b2ebf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa79402ade0 tx=0x7fa794005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814b29d5562466bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.352+0000 7fa788ff9640 1 -- 172.21.15.110:0/1151867652 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7940316d0 con 0x55f105b2e7d0 2024-04-25T00:15:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.353+0000 7fa788ff9640 1 -- 172.21.15.110:0/1151867652 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa794036030 con 0x55f105b2e7d0 2024-04-25T00:15:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.353+0000 7fa788ff9640 1 -- 172.21.15.110:0/1151867652 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794004480 con 0x55f105b2e7d0 2024-04-25T00:15:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.353+0000 7fa78b7fe640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7840a2c80 0x7fa7840a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.353+0000 7fa79a59c640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7840a2c80 0x7fa7840a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa78b7fe640 1 -- 172.21.15.110:0/955606402 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa7900cf120 con 0x7fa79c102800 2024-04-25T00:15:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa7a383e740 1 -- 172.21.15.110:0/1151867652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2e7d0 msgr2=0x55f105b2ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa7a383e740 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2e7d0 0x55f105b2ebf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa79402ade0 tx=0x7fa794005fa0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa7a383e740 1 -- 172.21.15.110:0/1151867652 shutdown_connections 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa7a383e740 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b34330 0x55f105b367c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa7a383e740 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2f1c0 0x55f105b33cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa7a383e740 1 --2- 172.21.15.110:0/1151867652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2e7d0 0x55f105b2ebf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa7a383e740 1 -- 172.21.15.110:0/1151867652 >> 172.21.15.110:0/1151867652 conn(0x55f105b21630 msgr2=0x55f105b21a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.354+0000 7fa79a59c640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7840a2c80 0x7fa7840a5140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55f105b2f5c0 tx=0x7fa78c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 -- 172.21.15.110:0/1151867652 shutdown_connections 2024-04-25T00:15:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 -- 172.21.15.110:0/1151867652 wait complete. 2024-04-25T00:15:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 -- start start 2024-04-25T00:15:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 Processor -- start 2024-04-25T00:15:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2e7d0 0x55f105b38aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2f1c0 0x55f105b38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b34330 0x55f105b40020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f105b37100 con 0x55f105b2e7d0 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f105a45b50 con 0x55f105b34330 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.355+0000 7fa7a383e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f105b37280 con 0x55f105b2f1c0 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa7a383e740 10 client.0 _ll_register_callbacks cb 0x55f105a80ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2f1c0 0x55f105b38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2f1c0 0x55f105b38fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:38364/0 (socket says 172.21.15.110:38364) 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 -- 172.21.15.110:0/2107576051 learned_addr learned my addr 172.21.15.110:0/2107576051 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa79ad9d640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b34330 0x55f105b40020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa79a59c640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2e7d0 0x55f105b38aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 -- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b34330 msgr2=0x55f105b40020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b34330 0x55f105b40020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 -- 172.21.15.110:0/2107576051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2e7d0 msgr2=0x55f105b38aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2e7d0 0x55f105b38aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 -- 172.21.15.110:0/2107576051 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f10586a620 con 0x55f105b2f1c0 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa79a59c640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2e7d0 0x55f105b38aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa79ad9d640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b34330 0x55f105b40020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa799d9b640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2f1c0 0x55f105b38fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa790009920 tx=0x7fa790009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa5f2e95f467374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.356+0000 7fa781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa781ffb640 1 -- 172.21.15.110:0/2107576051 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79009b490 con 0x55f105b2f1c0 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa781ffb640 1 -- 172.21.15.110:0/2107576051 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa790097fc0 con 0x55f105b2f1c0 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa7a383e740 1 -- 172.21.15.110:0/2107576051 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa794036030 con 0x55f105b2f1c0 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa781ffb640 1 -- 172.21.15.110:0/2107576051 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa79009b2b0 con 0x55f105b2f1c0 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa7a383e740 10 client.25846 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa7a383e740 1 -- 172.21.15.110:0/2107576051 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f105b43490 con 0x55f105b2f1c0 2024-04-25T00:15:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa781ffb640 1 -- 172.21.15.110:0/2107576051 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa794036030 con 0x55f105b2f1c0 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa781ffb640 1 -- 172.21.15.110:0/2107576051 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fa7900d2c90 con 0x55f105b2f1c0 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa7a383e740 10 client.25846 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa7a383e740 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.357+0000 7fa7a383e740 20 client.25846 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.358+0000 7fa7a383e740 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f105b44b80 0x55f105b47000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.358+0000 7fa7a383e740 4 client.25846 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.358+0000 7fa7a383e740 1 -- 172.21.15.110:0/2107576051 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f105a971f0 con 0x55f105b44b80 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.358+0000 7fa79a59c640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f105b44b80 0x55f105b47000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.359+0000 7fa79a59c640 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f105b44b80 0x55f105b47000 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723193ad60e83608 server_cookie=3c240f50d643a167 in_seq=0 out_seq=0 2024-04-25T00:15:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.359+0000 7fa781ffb640 10 client.25846 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.359+0000 7fa781ffb640 1 -- 172.21.15.110:0/2107576051 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f105a971f0 con 0x55f105b44b80 2024-04-25T00:15:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.359+0000 7fa781ffb640 10 client.25846 handle_command_reply: tid=0 2024-04-25T00:15:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.359+0000 7fa7a383e740 1 client.25846 shutdown 2024-04-25T00:15:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 -- 172.21.15.110:0/2107576051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2f1c0 msgr2=0x55f105b38fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2f1c0 0x55f105b38fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa790009920 tx=0x7fa790009510 comp rx=0 tx=0).stop 2024-04-25T00:15:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 -- 172.21.15.110:0/2107576051 shutdown_connections 2024-04-25T00:15:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f105b44b80 0x55f105b47000 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b34330 0x55f105b40020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b2f1c0 0x55f105b38fe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 --2- 172.21.15.110:0/2107576051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2e7d0 0x55f105b38aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa781ffb640 0 client.25846 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 -- 172.21.15.110:0/2107576051 >> 172.21.15.110:0/2107576051 conn(0x55f105b21630 msgr2=0x55f105b20830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 -- 172.21.15.110:0/2107576051 shutdown_connections 2024-04-25T00:15:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 1 -- 172.21.15.110:0/2107576051 wait complete. 2024-04-25T00:15:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.360+0000 7fa7a383e740 20 client.25846 trim_cache size 0 max 16384 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.381+0000 7fa7a383e740 1 Processor -- start 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.381+0000 7fa7a383e740 1 -- start start 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.381+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.381+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.381+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 0x55f105b24be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.381+0000 7fa7a383e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c010c10 con 0x55f105b2f1c0 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.381+0000 7fa7a383e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c00e4e0 con 0x55f105b2e7d0 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.381+0000 7fa7a383e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c00c330 con 0x55f105b34330 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:50364/0 (socket says 172.21.15.110:50364) 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79ad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 0x55f105b24be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 -- 172.21.15.110:0/3339723925 learned_addr learned my addr 172.21.15.110:0/3339723925 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41786/0 (socket says 172.21.15.110:41786) 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 -- 172.21.15.110:0/3339723925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 msgr2=0x55f105b24be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 --2- 172.21.15.110:0/3339723925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 0x55f105b24be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 -- 172.21.15.110:0/3339723925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 msgr2=0x55f105b3ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 --2- 172.21.15.110:0/3339723925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 -- 172.21.15.110:0/3339723925 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f105b47680 con 0x55f105b2e7d0 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa79a59c640 1 --2- 172.21.15.110:0/3339723925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b246a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa78c016040 tx=0x7fa78c018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda03748cc9e2c65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa783fff640 1 -- 172.21.15.110:0/3339723925 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c030070 con 0x55f105b2e7d0 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa783fff640 1 -- 172.21.15.110:0/3339723925 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa78c01b040 con 0x55f105b2e7d0 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.382+0000 7fa783fff640 1 -- 172.21.15.110:0/3339723925 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c02a710 con 0x55f105b2e7d0 2024-04-25T00:15:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 -- 172.21.15.110:0/3339723925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 msgr2=0x55f105b246a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3339723925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b246a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa78c016040 tx=0x7fa78c018000 comp rx=0 tx=0).stop 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 -- 172.21.15.110:0/3339723925 shutdown_connections 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3339723925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 0x55f105b24be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3339723925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3339723925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b246a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 -- 172.21.15.110:0/3339723925 >> 172.21.15.110:0/3339723925 conn(0x55f105b21630 msgr2=0x55f105b21a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 -- 172.21.15.110:0/3339723925 shutdown_connections 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.383+0000 7fa7a383e740 1 -- 172.21.15.110:0/3339723925 wait complete. 2024-04-25T00:15:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 1 -- start start 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 1 Processor -- start 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 0x55f105b3ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c00c9a0 con 0x55f105b2f1c0 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c00de10 con 0x55f105b2e7d0 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c00d070 con 0x55f105b34330 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa7a383e740 10 client.0 _ll_register_callbacks cb 0x7fa79000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa79a59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa79a59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b3da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:50368/0 (socket says 172.21.15.110:50368) 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa79a59c640 1 -- 172.21.15.110:0/3894700986 learned_addr learned my addr 172.21.15.110:0/3894700986 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa799d9b640 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.384+0000 7fa79ad9d640 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 0x55f105b3ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa79a59c640 1 -- 172.21.15.110:0/3894700986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 msgr2=0x55f105b3ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa79a59c640 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 0x55f105b3ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa79a59c640 1 -- 172.21.15.110:0/3894700986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 msgr2=0x55f105b3df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa79a59c640 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa79a59c640 1 -- 172.21.15.110:0/3894700986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f105b3f550 con 0x55f105b2e7d0 2024-04-25T00:15:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa79a59c640 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b3da10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0092b0 tx=0x7fa78c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da3af6ff417b87d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:15:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa781ffb640 1 -- 172.21.15.110:0/3894700986 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c03b060 con 0x55f105b2e7d0 2024-04-25T00:15:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa781ffb640 1 -- 172.21.15.110:0/3894700986 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa78c029040 con 0x55f105b2e7d0 2024-04-25T00:15:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa7a383e740 1 -- 172.21.15.110:0/3894700986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa78c01b040 con 0x55f105b2e7d0 2024-04-25T00:15:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa781ffb640 1 -- 172.21.15.110:0/3894700986 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa78c030030 con 0x55f105b2e7d0 2024-04-25T00:15:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa7a383e740 10 client.25868 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.385+0000 7fa7a383e740 1 -- 172.21.15.110:0/3894700986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f105bab460 con 0x55f105b2e7d0 2024-04-25T00:15:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa781ffb640 1 -- 172.21.15.110:0/3894700986 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa78c01b040 con 0x55f105b2e7d0 2024-04-25T00:15:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa781ffb640 1 -- 172.21.15.110:0/3894700986 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fa78c020040 con 0x55f105b2e7d0 2024-04-25T00:15:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa7a383e740 10 client.25868 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa7a383e740 10 client.25868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa7a383e740 20 client.25868 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f105babd80 0x55f105bae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa7a383e740 4 client.25868 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:15:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa7a383e740 1 -- 172.21.15.110:0/3894700986 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f105a91540 con 0x55f105babd80 2024-04-25T00:15:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.386+0000 7fa799d9b640 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f105babd80 0x55f105bae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.387+0000 7fa799d9b640 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f105babd80 0x55f105bae1a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14760cc752aa1ecf server_cookie=67ffe9cb049e289a in_seq=0 out_seq=0 2024-04-25T00:15:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.387+0000 7fa781ffb640 10 client.25868 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa781ffb640 1 -- 172.21.15.110:0/3894700986 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f105a91540 con 0x55f105babd80 2024-04-25T00:15:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa781ffb640 10 client.25868 handle_command_reply: tid=0 2024-04-25T00:15:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 client.25868 shutdown 2024-04-25T00:15:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 -- 172.21.15.110:0/3894700986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 msgr2=0x55f105b3da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b3da10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0092b0 tx=0x7fa78c009510 comp rx=0 tx=0).stop 2024-04-25T00:15:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 -- 172.21.15.110:0/3894700986 shutdown_connections 2024-04-25T00:15:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f105babd80 0x55f105bae1a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa781ffb640 0 client.25868 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f105b34330 0x55f105b3ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f105b2f1c0 0x55f105b3df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 --2- 172.21.15.110:0/3894700986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f105b2e7d0 0x55f105b3da10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 -- 172.21.15.110:0/3894700986 >> 172.21.15.110:0/3894700986 conn(0x55f105b21630 msgr2=0x55f105b246a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 -- 172.21.15.110:0/3894700986 shutdown_connections 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 1 -- 172.21.15.110:0/3894700986 wait complete. 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.403+0000 7fa7a383e740 20 client.25868 trim_cache size 0 max 16384 2024-04-25T00:15:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:15:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:15:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:15:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:15:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7840a2c80 msgr2=0x7fa7840a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7840a2c80 0x7fa7840a5140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55f105b2f5c0 tx=0x7fa78c009110 comp rx=0 tx=0).stop 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa79c102800 msgr2=0x7fa79c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa79c102800 0x7fa79c199f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa79000ea60 tx=0x7fa79000ee60 comp rx=0 tx=0).stop 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 shutdown_connections 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa7840a2c80 0x7fa7840a5140 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa79c1031c0 0x7fa79c1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa79c102800 0x7fa79c199f90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 --2- 172.21.15.110:0/955606402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c101610 0x7fa79c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 >> 172.21.15.110:0/955606402 conn(0x7fa79c0fcba0 msgr2=0x7fa79c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 shutdown_connections 2024-04-25T00:15:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:06.404+0000 7fa7a0b30640 1 -- 172.21.15.110:0/955606402 wait complete. 2024-04-25T00:15:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 720.0 s, 826.8 tps, lat 120.642 ms stddev 18.265 2024-04-25T00:15:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:07 smithi173 ceph-mon[32402]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 790 op/s 2024-04-25T00:15:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:07 smithi188 ceph-mon[33285]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 790 op/s 2024-04-25T00:15:08.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:07 smithi110 ceph-mon[29187]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 790 op/s 2024-04-25T00:15:10.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:09 smithi188 ceph-mon[33285]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 993 op/s 2024-04-25T00:15:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:09 smithi173 ceph-mon[32402]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 993 op/s 2024-04-25T00:15:10.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:09 smithi110 ceph-mon[29187]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 993 op/s 2024-04-25T00:15:11.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:10 smithi188 ceph-mon[33285]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.9 MiB/s wr, 768 op/s 2024-04-25T00:15:11.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:10 smithi110 ceph-mon[29187]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.9 MiB/s wr, 768 op/s 2024-04-25T00:15:11.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:10 smithi173 ceph-mon[32402]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.9 MiB/s wr, 768 op/s 2024-04-25T00:15:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 725.0 s, 837.8 tps, lat 119.634 ms stddev 20.513 2024-04-25T00:15:14.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:13 smithi173 ceph-mon[32402]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 968 op/s 2024-04-25T00:15:14.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:13 smithi188 ceph-mon[33285]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 968 op/s 2024-04-25T00:15:14.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:13 smithi110 ceph-mon[29187]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 968 op/s 2024-04-25T00:15:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:15 smithi173 ceph-mon[32402]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 856 op/s 2024-04-25T00:15:16.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:15 smithi188 ceph-mon[33285]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 856 op/s 2024-04-25T00:15:16.289 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:15 smithi110 ceph-mon[29187]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 856 op/s 2024-04-25T00:15:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 730.0 s, 834.4 tps, lat 120.085 ms stddev 20.963 2024-04-25T00:15:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:17 smithi173 ceph-mon[32402]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 756 op/s 2024-04-25T00:15:18.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:17 smithi188 ceph-mon[33285]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 756 op/s 2024-04-25T00:15:18.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:17 smithi110 ceph-mon[29187]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 756 op/s 2024-04-25T00:15:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 -- 172.21.15.110:0/3189150251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920102990 msgr2=0x7fc920102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 --2- 172.21.15.110:0/3189150251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920102990 0x7fc920102e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009860 tx=0x7fc90c02a340 comp rx=0 tx=0).stop 2024-04-25T00:15:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 -- 172.21.15.110:0/3189150251 shutdown_connections 2024-04-25T00:15:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 --2- 172.21.15.110:0/3189150251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc920103350 0x7fc920109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 --2- 172.21.15.110:0/3189150251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920102990 0x7fc920102e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 --2- 172.21.15.110:0/3189150251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9201017a0 0x7fc920101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 -- 172.21.15.110:0/3189150251 >> 172.21.15.110:0/3189150251 conn(0x7fc9200fcd30 msgr2=0x7fc9200ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 -- 172.21.15.110:0/3189150251 shutdown_connections 2024-04-25T00:15:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.535+0000 7fc926eb3640 1 -- 172.21.15.110:0/3189150251 wait complete. 2024-04-25T00:15:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.536+0000 7fc926eb3640 1 Processor -- start 2024-04-25T00:15:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.536+0000 7fc926eb3640 1 -- start start 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc926eb3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9201017a0 0x7fc9201978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc926eb3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc920102990 0x7fc920197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc926eb3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920103350 0x7fc92019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc926eb3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc92010d2b0 con 0x7fc920102990 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc926eb3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc92010d130 con 0x7fc9201017a0 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc926eb3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc92010d430 con 0x7fc920103350 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc925450640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920103350 0x7fc92019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc924c4f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9201017a0 0x7fc9201978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc924c4f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9201017a0 0x7fc9201978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:36792/0 (socket says 172.21.15.110:36792) 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc924c4f640 1 -- 172.21.15.110:0/862416888 learned_addr learned my addr 172.21.15.110:0/862416888 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc925450640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920103350 0x7fc92019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:33328/0 (socket says 172.21.15.110:33328) 2024-04-25T00:15:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc91ffff640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc920102990 0x7fc920197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc925450640 1 -- 172.21.15.110:0/862416888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9201017a0 msgr2=0x7fc9201978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc925450640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9201017a0 0x7fc9201978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc925450640 1 -- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc920102990 msgr2=0x7fc920197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc925450640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc920102990 0x7fc920197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.537+0000 7fc925450640 1 -- 172.21.15.110:0/862416888 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92019f4f0 con 0x7fc920103350 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.538+0000 7fc925450640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920103350 0x7fc92019ee80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc918009fd0 tx=0x7fc91800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.538+0000 7fc91dffb640 1 -- 172.21.15.110:0/862416888 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91800ce20 con 0x7fc920103350 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.538+0000 7fc926eb3640 1 -- 172.21.15.110:0/862416888 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92019f7e0 con 0x7fc920103350 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.538+0000 7fc91dffb640 1 -- 172.21.15.110:0/862416888 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc918005020 con 0x7fc920103350 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.538+0000 7fc91dffb640 1 -- 172.21.15.110:0/862416888 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918010690 con 0x7fc920103350 2024-04-25T00:15:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.538+0000 7fc926eb3640 1 -- 172.21.15.110:0/862416888 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92019fd00 con 0x7fc920103350 2024-04-25T00:15:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.542+0000 7fc929bbe740 1 Processor -- start 2024-04-25T00:15:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.543+0000 7fc91dffb640 1 -- 172.21.15.110:0/862416888 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc9180051d0 con 0x7fc920103350 2024-04-25T00:15:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.543+0000 7fc929bbe740 1 -- start start 2024-04-25T00:15:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726ae50 0x55b74726b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726b840 0x55b747270340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 0x55b747272e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc929bbe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b7471cecb0 con 0x55b74726b840 2024-04-25T00:15:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc929bbe740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b7471775b0 con 0x55b7472709b0 2024-04-25T00:15:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc929bbe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b7471cac70 con 0x55b74726ae50 2024-04-25T00:15:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc91dffb640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9100a2c30 0x7fc9100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc91dffb640 1 -- 172.21.15.110:0/862416888 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc9180cf6e0 con 0x7fc920103350 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc925450640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9100a2c30 0x7fc9100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc924c4f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726b840 0x55b747270340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc924c4f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726b840 0x55b747270340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49466/0 (socket says 172.21.15.110:49466) 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.544+0000 7fc924c4f640 1 -- 172.21.15.110:0/1056721032 learned_addr learned my addr 172.21.15.110:0/1056721032 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.545+0000 7fc924c4f640 1 -- 172.21.15.110:0/1056721032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726ae50 msgr2=0x55b74726b270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.545+0000 7fc924c4f640 1 --2- 172.21.15.110:0/1056721032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726ae50 0x55b74726b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.545+0000 7fc924c4f640 1 -- 172.21.15.110:0/1056721032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 msgr2=0x55b747272e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.545+0000 7fc924c4f640 1 --2- 172.21.15.110:0/1056721032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 0x55b747272e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.545+0000 7fc925450640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9100a2c30 0x7fc9100a50f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc91809bf00 tx=0x7fc918005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.545+0000 7fc924c4f640 1 -- 172.21.15.110:0/1056721032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b746ee3210 con 0x55b74726b840 2024-04-25T00:15:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.545+0000 7fc924c4f640 1 --2- 172.21.15.110:0/1056721032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726b840 0x55b747270340 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc914009d90 tx=0x7fc914011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7c76c996c4c8d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.546+0000 7fc90affd640 1 -- 172.21.15.110:0/1056721032 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914004220 con 0x55b74726b840 2024-04-25T00:15:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.546+0000 7fc90affd640 1 -- 172.21.15.110:0/1056721032 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9140043c0 con 0x55b74726b840 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.546+0000 7fc90affd640 1 -- 172.21.15.110:0/1056721032 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9140198f0 con 0x55b74726b840 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 -- 172.21.15.110:0/1056721032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726b840 msgr2=0x55b747270340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 --2- 172.21.15.110:0/1056721032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726b840 0x55b747270340 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc914009d90 tx=0x7fc914011810 comp rx=0 tx=0).stop 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 -- 172.21.15.110:0/1056721032 shutdown_connections 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 --2- 172.21.15.110:0/1056721032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 0x55b747272e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 --2- 172.21.15.110:0/1056721032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726b840 0x55b747270340 secure :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc914009d90 tx=0x7fc914011810 comp rx=0 tx=0).stop 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 --2- 172.21.15.110:0/1056721032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726ae50 0x55b74726b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 -- 172.21.15.110:0/1056721032 >> 172.21.15.110:0/1056721032 conn(0x55b7472664c0 msgr2=0x55b7472668c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 -- 172.21.15.110:0/1056721032 shutdown_connections 2024-04-25T00:15:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 -- 172.21.15.110:0/1056721032 wait complete. 2024-04-25T00:15:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.547+0000 7fc929bbe740 1 -- start start 2024-04-25T00:15:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc929bbe740 1 Processor -- start 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7472709b0 0x55b74726e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726f7c0 0x55b74726fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc929bbe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b747273740 con 0x55b74726ae50 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc929bbe740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b74718ab50 con 0x55b74726f7c0 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc929bbe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b7472738c0 con 0x55b7472709b0 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc929bbe740 10 client.0 _ll_register_callbacks cb 0x55b7471c5ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc91ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc924c4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7472709b0 0x55b74726e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc91ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49470/0 (socket says 172.21.15.110:49470) 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc91ffff640 1 -- 172.21.15.110:0/861590268 learned_addr learned my addr 172.21.15.110:0/861590268 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.548+0000 7fc91ffff640 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726f7c0 0x55b74726fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc924c4f640 1 -- 172.21.15.110:0/861590268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726f7c0 msgr2=0x55b74726fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc924c4f640 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726f7c0 0x55b74726fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc924c4f640 1 -- 172.21.15.110:0/861590268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 msgr2=0x55b74726e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc924c4f640 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc924c4f640 1 -- 172.21.15.110:0/861590268 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b746faf620 con 0x55b7472709b0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc924c4f640 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7472709b0 0x55b74726e800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55b74726bd00 tx=0x7fc914005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174ecbf28f246528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc8fffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc8fffff640 1 -- 172.21.15.110:0/861590268 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914027070 con 0x55b7472709b0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc929bbe740 1 -- 172.21.15.110:0/861590268 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9140043c0 con 0x55b7472709b0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc8fffff640 1 -- 172.21.15.110:0/861590268 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc9140035f0 con 0x55b7472709b0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc8fffff640 1 -- 172.21.15.110:0/861590268 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc914033050 con 0x55b7472709b0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.549+0000 7fc929bbe740 10 client.25861 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc929bbe740 1 -- 172.21.15.110:0/861590268 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b7472889a0 con 0x55b7472709b0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc8fffff640 1 -- 172.21.15.110:0/861590268 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9140043c0 con 0x55b7472709b0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc8fffff640 1 -- 172.21.15.110:0/861590268 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc914004c20 con 0x55b7472709b0 2024-04-25T00:15:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc929bbe740 10 client.25861 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc929bbe740 10 client.25861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc929bbe740 20 client.25861 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc929bbe740 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b747289e10 0x55b74728c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc929bbe740 4 client.25861 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc929bbe740 1 -- 172.21.15.110:0/861590268 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7471dc1f0 con 0x55b747289e10 2024-04-25T00:15:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.550+0000 7fc91ffff640 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b747289e10 0x55b74728c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.551+0000 7fc8fffff640 10 client.25861 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.551+0000 7fc91ffff640 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b747289e10 0x55b74728c200 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f57f20d4048fc73c server_cookie=8dd47a1d39ea8068 in_seq=0 out_seq=0 2024-04-25T00:15:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.551+0000 7fc8fffff640 1 -- 172.21.15.110:0/861590268 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7471dc1f0 con 0x55b747289e10 2024-04-25T00:15:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.551+0000 7fc8fffff640 10 client.25861 handle_command_reply: tid=0 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 client.25861 shutdown 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 -- 172.21.15.110:0/861590268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7472709b0 msgr2=0x55b74726e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7472709b0 0x55b74726e800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55b74726bd00 tx=0x7fc914005040 comp rx=0 tx=0).stop 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 -- 172.21.15.110:0/861590268 shutdown_connections 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b747289e10 0x55b74728c200 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726f7c0 0x55b74726fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7472709b0 0x55b74726e800 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 --2- 172.21.15.110:0/861590268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc8fffff640 0 client.25861 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 -- 172.21.15.110:0/861590268 >> 172.21.15.110:0/861590268 conn(0x55b7472664c0 msgr2=0x55b7472655c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 -- 172.21.15.110:0/861590268 shutdown_connections 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 1 -- 172.21.15.110:0/861590268 wait complete. 2024-04-25T00:15:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.552+0000 7fc929bbe740 20 client.25861 trim_cache size 0 max 16384 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.576+0000 7fc929bbe740 1 Processor -- start 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.576+0000 7fc929bbe740 1 -- start start 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.576+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.576+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b74726c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.576+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 0x55b74726ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.576+0000 7fc929bbe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201a9070 con 0x55b74726ae50 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.576+0000 7fc929bbe740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201a6940 con 0x55b7472709b0 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.576+0000 7fc929bbe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201a4790 con 0x55b74726f7c0 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b74726c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b74726c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:33348/0 (socket says 172.21.15.110:33348) 2024-04-25T00:15:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 -- 172.21.15.110:0/3261196800 learned_addr learned my addr 172.21.15.110:0/3261196800 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc91ffff640 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 0x55b74726ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc91ffff640 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 -- 172.21.15.110:0/3261196800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 msgr2=0x55b74726ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 0x55b74726ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 -- 172.21.15.110:0/3261196800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 msgr2=0x55b74726bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 -- 172.21.15.110:0/3261196800 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7472608e0 con 0x55b74726f7c0 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc924c4f640 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b74726c940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9140058c0 tx=0x7fc9140092c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478b8be996f03dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc90a7fc640 1 -- 172.21.15.110:0/3261196800 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914030070 con 0x55b74726f7c0 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc90a7fc640 1 -- 172.21.15.110:0/3261196800 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc914019430 con 0x55b74726f7c0 2024-04-25T00:15:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.577+0000 7fc90a7fc640 1 -- 172.21.15.110:0/3261196800 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914027070 con 0x55b74726f7c0 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 -- 172.21.15.110:0/3261196800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 msgr2=0x55b74726c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b74726c940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9140058c0 tx=0x7fc9140092c0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 -- 172.21.15.110:0/3261196800 shutdown_connections 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7472709b0 0x55b74726ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b74726c940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 --2- 172.21.15.110:0/3261196800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b74726ae50 0x55b74726bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 -- 172.21.15.110:0/3261196800 >> 172.21.15.110:0/3261196800 conn(0x55b7472664c0 msgr2=0x55b747273d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 -- 172.21.15.110:0/3261196800 shutdown_connections 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.579+0000 7fc929bbe740 1 -- 172.21.15.110:0/3261196800 wait complete. 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.580+0000 7fc929bbe740 1 -- start start 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.580+0000 7fc929bbe740 1 Processor -- start 2024-04-25T00:15:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.580+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726ae50 0x55b7472862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc929bbe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b747282d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc91ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726ae50 0x55b7472862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc91ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726ae50 0x55b7472862e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:36808/0 (socket says 172.21.15.110:36808) 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc91ffff640 1 -- 172.21.15.110:0/2725437527 learned_addr learned my addr 172.21.15.110:0/2725437527 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc924c4f640 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b747282d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc929bbe740 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7472709b0 0x55b7472833b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201a4e00 con 0x55b7472709b0 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201a6270 con 0x55b74726ae50 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201a54d0 con 0x55b74726f7c0 2024-04-25T00:15:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.581+0000 7fc929bbe740 10 client.0 _ll_register_callbacks cb 0x7fc914009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc91ffff640 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7472709b0 0x55b7472833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc924c4f640 1 -- 172.21.15.110:0/2725437527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726ae50 msgr2=0x55b7472862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc924c4f640 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726ae50 0x55b7472862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc924c4f640 1 -- 172.21.15.110:0/2725437527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7472709b0 msgr2=0x55b7472833b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc924c4f640 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7472709b0 0x55b7472833b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc924c4f640 1 -- 172.21.15.110:0/2725437527 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7472ee3b0 con 0x55b74726f7c0 2024-04-25T00:15:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc91ffff640 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726ae50 0x55b7472862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc924c4f640 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b747282d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc914019960 tx=0x7fc914003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535b3668c536d174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc8fffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc8fffff640 1 -- 172.21.15.110:0/2725437527 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914030040 con 0x55b74726f7c0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc8fffff640 1 -- 172.21.15.110:0/2725437527 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc914023870 con 0x55b74726f7c0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.582+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc914019430 con 0x55b74726f7c0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc8fffff640 1 -- 172.21.15.110:0/2725437527 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc914047070 con 0x55b74726f7c0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc929bbe740 10 client.25867 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b7472ef6f0 con 0x55b74726f7c0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc8fffff640 1 -- 172.21.15.110:0/2725437527 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc914003c80 con 0x55b74726f7c0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc8fffff640 1 -- 172.21.15.110:0/2725437527 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc91403f030 con 0x55b74726f7c0 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc929bbe740 10 client.25867 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc929bbe740 10 client.25867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc929bbe740 20 client.25867 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc929bbe740 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b7472f00c0 0x55b7472f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc929bbe740 4 client.25867 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:15:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc914019430 con 0x55b7472f00c0 2024-04-25T00:15:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.583+0000 7fc91ffff640 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b7472f00c0 0x55b7472f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.584+0000 7fc8fffff640 10 client.25867 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.584+0000 7fc91ffff640 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b7472f00c0 0x55b7472f24e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9224d5a917fd9f4d server_cookie=a38fbc6d54faf378 in_seq=0 out_seq=0 2024-04-25T00:15:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.608+0000 7fc8fffff640 1 -- 172.21.15.110:0/2725437527 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc914019430 con 0x55b7472f00c0 2024-04-25T00:15:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.608+0000 7fc8fffff640 10 client.25867 handle_command_reply: tid=0 2024-04-25T00:15:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.608+0000 7fc929bbe740 1 client.25867 shutdown 2024-04-25T00:15:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 msgr2=0x55b747282d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc929bbe740 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b747282d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc914019960 tx=0x7fc914003820 comp rx=0 tx=0).stop 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 shutdown_connections 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc929bbe740 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b7472f00c0 0x55b7472f24e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc8fffff640 0 client.25867 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc929bbe740 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7472709b0 0x55b7472833b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc929bbe740 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b74726f7c0 0x55b747282d80 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc914019960 tx=0x7fc914003820 comp rx=0 tx=0).stop 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc929bbe740 1 --2- 172.21.15.110:0/2725437527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b74726ae50 0x55b7472862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.609+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 >> 172.21.15.110:0/2725437527 conn(0x55b7472664c0 msgr2=0x55b747269240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.610+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 shutdown_connections 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.610+0000 7fc929bbe740 1 -- 172.21.15.110:0/2725437527 wait complete. 2024-04-25T00:15:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.610+0000 7fc929bbe740 20 client.25867 trim_cache size 0 max 16384 2024-04-25T00:15:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:15:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:15:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:15:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:15:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.612+0000 7fc926eb3640 1 -- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9100a2c30 msgr2=0x7fc9100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.612+0000 7fc926eb3640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9100a2c30 0x7fc9100a50f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc91809bf00 tx=0x7fc918005f70 comp rx=0 tx=0).stop 2024-04-25T00:15:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.612+0000 7fc926eb3640 1 -- 172.21.15.110:0/862416888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920103350 msgr2=0x7fc92019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.612+0000 7fc926eb3640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920103350 0x7fc92019ee80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc918009fd0 tx=0x7fc91800ef10 comp rx=0 tx=0).stop 2024-04-25T00:15:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.615+0000 7fc926eb3640 1 -- 172.21.15.110:0/862416888 shutdown_connections 2024-04-25T00:15:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.615+0000 7fc926eb3640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc9100a2c30 0x7fc9100a50f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.616+0000 7fc926eb3640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc920103350 0x7fc92019ee80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.616+0000 7fc926eb3640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc920102990 0x7fc920197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.616+0000 7fc926eb3640 1 --2- 172.21.15.110:0/862416888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9201017a0 0x7fc9201978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.616+0000 7fc926eb3640 1 -- 172.21.15.110:0/862416888 >> 172.21.15.110:0/862416888 conn(0x7fc9200fcd30 msgr2=0x7fc920105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.616+0000 7fc926eb3640 1 -- 172.21.15.110:0/862416888 shutdown_connections 2024-04-25T00:15:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:19.616+0000 7fc926eb3640 1 -- 172.21.15.110:0/862416888 wait complete. 2024-04-25T00:15:20.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:19 smithi173 ceph-mon[32402]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 962 op/s 2024-04-25T00:15:20.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:19 smithi188 ceph-mon[33285]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 962 op/s 2024-04-25T00:15:20.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:19 smithi110 ceph-mon[29187]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 962 op/s 2024-04-25T00:15:21.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:20 smithi188 ceph-mon[33285]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 775 op/s 2024-04-25T00:15:21.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:20 smithi110 ceph-mon[29187]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 775 op/s 2024-04-25T00:15:21.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:20 smithi173 ceph-mon[32402]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 775 op/s 2024-04-25T00:15:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 735.0 s, 829.8 tps, lat 120.439 ms stddev 19.855 2024-04-25T00:15:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:23 smithi173 ceph-mon[32402]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.7 MiB/s wr, 983 op/s 2024-04-25T00:15:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:23 smithi188 ceph-mon[33285]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.7 MiB/s wr, 983 op/s 2024-04-25T00:15:24.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:23 smithi110 ceph-mon[29187]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.7 MiB/s wr, 983 op/s 2024-04-25T00:15:26.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:25 smithi173 ceph-mon[32402]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 858 op/s 2024-04-25T00:15:26.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:25 smithi188 ceph-mon[33285]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 858 op/s 2024-04-25T00:15:26.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:25 smithi110 ceph-mon[29187]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 858 op/s 2024-04-25T00:15:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 740.0 s, 828.6 tps, lat 120.337 ms stddev 14.736 2024-04-25T00:15:28.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:27 smithi173 ceph-mon[32402]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-04-25T00:15:28.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:27 smithi188 ceph-mon[33285]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-04-25T00:15:28.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:27 smithi110 ceph-mon[29187]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-04-25T00:15:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:28 smithi188 ceph-mon[33285]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 938 op/s 2024-04-25T00:15:29.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:28 smithi110 ceph-mon[29187]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 938 op/s 2024-04-25T00:15:29.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:28 smithi173 ceph-mon[32402]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.1 MiB/s wr, 938 op/s 2024-04-25T00:15:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 745.0 s, 821.6 tps, lat 122.131 ms stddev 15.520 2024-04-25T00:15:32.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:31 smithi173 ceph-mon[32402]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 867 op/s 2024-04-25T00:15:32.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:31 smithi188 ceph-mon[33285]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 867 op/s 2024-04-25T00:15:32.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:31 smithi110 ceph-mon[29187]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 867 op/s 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 -- 172.21.15.110:0/4233350326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc101880 msgr2=0x7f57fc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 --2- 172.21.15.110:0/4233350326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc101880 0x7f57fc101d00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009900 tx=0x7f57f002a400 comp rx=0 tx=0).stop 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 -- 172.21.15.110:0/4233350326 shutdown_connections 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 --2- 172.21.15.110:0/4233350326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc102240 0x7f57fc108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 --2- 172.21.15.110:0/4233350326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc101880 0x7f57fc101d00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 --2- 172.21.15.110:0/4233350326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57fc100690 0x7f57fc100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 -- 172.21.15.110:0/4233350326 >> 172.21.15.110:0/4233350326 conn(0x7f57fc0fbc00 msgr2=0x7f57fc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 -- 172.21.15.110:0/4233350326 shutdown_connections 2024-04-25T00:15:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.747+0000 7f5802329640 1 -- 172.21.15.110:0/4233350326 wait complete. 2024-04-25T00:15:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.748+0000 7f5802329640 1 Processor -- start 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.748+0000 7f5802329640 1 -- start start 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.748+0000 7f5802329640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc100690 0x7f57fc06aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.748+0000 7f5802329640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57fc101880 0x7f57fc19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.748+0000 7f5802329640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc102240 0x7f57fc1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.748+0000 7f5802329640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc10c1a0 con 0x7f57fc101880 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.748+0000 7f5802329640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc10c020 con 0x7f57fc102240 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.748+0000 7f5802329640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc10c320 con 0x7f57fc100690 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f58008c6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc102240 0x7f57fc1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc100690 0x7f57fc06aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc100690 0x7f57fc06aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:39204/0 (socket says 172.21.15.110:39204) 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 -- 172.21.15.110:0/41355521 learned_addr learned my addr 172.21.15.110:0/41355521 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fb7fe640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57fc101880 0x7f57fc19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 -- 172.21.15.110:0/41355521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc102240 msgr2=0x7f57fc1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc102240 0x7f57fc1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 -- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57fc101880 msgr2=0x7f57fc19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57fc101880 0x7f57fc19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 -- 172.21.15.110:0/41355521 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc1a17a0 con 0x7f57fc100690 2024-04-25T00:15:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.749+0000 7f57fbfff640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc100690 0x7f57fc06aae0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e9c0 tx=0x7f57ec00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.750+0000 7f57f97fa640 1 -- 172.21.15.110:0/41355521 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00cd80 con 0x7f57fc100690 2024-04-25T00:15:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.750+0000 7f57f97fa640 1 -- 172.21.15.110:0/41355521 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ec0035f0 con 0x7f57fc100690 2024-04-25T00:15:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.750+0000 7f5802329640 1 -- 172.21.15.110:0/41355521 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc1a1a30 con 0x7f57fc100690 2024-04-25T00:15:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.750+0000 7f57f97fa640 1 -- 172.21.15.110:0/41355521 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec010690 con 0x7f57fc100690 2024-04-25T00:15:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.750+0000 7f5802329640 1 -- 172.21.15.110:0/41355521 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc1a1f50 con 0x7f57fc100690 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f57f97fa640 1 -- 172.21.15.110:0/41355521 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f57ec0037a0 con 0x7f57fc100690 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f5805038740 1 Processor -- start 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f5805038740 1 -- start start 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f5805038740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eade50 0x5599f0eae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f5805038740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eae840 0x5599f0eb3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f5805038740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0eb5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f5805038740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5599f0e11cb0 con 0x5599f0eade50 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f5805038740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5599f0dba5b0 con 0x5599f0eae840 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f5805038740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599f0e0dc70 con 0x5599f0eb39b0 2024-04-25T00:15:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f58008c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eade50 0x5599f0eae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eae840 0x5599f0eb3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.753+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eae840 0x5599f0eb3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:40154/0 (socket says 172.21.15.110:40154) 2024-04-25T00:15:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57fbfff640 1 -- 172.21.15.110:0/2074578890 learned_addr learned my addr 172.21.15.110:0/2074578890 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57fb7fe640 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0eb5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57fbfff640 1 -- 172.21.15.110:0/2074578890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 msgr2=0x5599f0eb5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57fbfff640 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0eb5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57fbfff640 1 -- 172.21.15.110:0/2074578890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eade50 msgr2=0x5599f0eae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57fbfff640 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eade50 0x5599f0eae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57fbfff640 1 -- 172.21.15.110:0/2074578890 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599f0b26210 con 0x5599f0eae840 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57f97fa640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f57e40a2ce0 0x7f57e40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f58008c6640 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eade50 0x5599f0eae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f58008c6640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f57e40a2ce0 0x7f57e40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57f97fa640 1 -- 172.21.15.110:0/41355521 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f57ec0cf390 con 0x7f57fc100690 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57fbfff640 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eae840 0x5599f0eb3340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0d7390 tx=0x7f57ec097ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf2f489db60b1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.754+0000 7f57e27fc640 1 -- 172.21.15.110:0/2074578890 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec09b9f0 con 0x5599f0eae840 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.755+0000 7f57e27fc640 1 -- 172.21.15.110:0/2074578890 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ec09eda0 con 0x5599f0eae840 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.755+0000 7f57e27fc640 1 -- 172.21.15.110:0/2074578890 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec09eff0 con 0x5599f0eae840 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.755+0000 7f5805038740 1 -- 172.21.15.110:0/2074578890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eae840 msgr2=0x5599f0eb3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.755+0000 7f58008c6640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f57e40a2ce0 0x7f57e40a51a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5599f0eaf0a0 tx=0x7f57f4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.755+0000 7f5805038740 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eae840 0x5599f0eb3340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0d7390 tx=0x7f57ec097ec0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 -- 172.21.15.110:0/2074578890 shutdown_connections 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0eb5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eae840 0x5599f0eb3340 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 --2- 172.21.15.110:0/2074578890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eade50 0x5599f0eae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 -- 172.21.15.110:0/2074578890 >> 172.21.15.110:0/2074578890 conn(0x5599f0ea94c0 msgr2=0x5599f0ea98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 -- 172.21.15.110:0/2074578890 shutdown_connections 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 -- 172.21.15.110:0/2074578890 wait complete. 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 -- start start 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 Processor -- start 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 0x5599f0eb8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0eb8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0ebf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5599f0eb68a0 con 0x5599f0eae840 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5599f0dcdb50 con 0x5599f0eade50 2024-04-25T00:15:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5599f0eb6a20 con 0x5599f0eb39b0 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f5805038740 10 client.0 _ll_register_callbacks cb 0x5599f0e08ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f58008c6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0ebf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 0x5599f0eb8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.756+0000 7f58008c6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0ebf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:39226/0 (socket says 172.21.15.110:39226) 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f58008c6640 1 -- 172.21.15.110:0/565614056 learned_addr learned my addr 172.21.15.110:0/565614056 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f57fb7fe640 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0eb8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f58008c6640 1 -- 172.21.15.110:0/565614056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 msgr2=0x5599f0eb8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f58008c6640 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 0x5599f0eb8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f58008c6640 1 -- 172.21.15.110:0/565614056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 msgr2=0x5599f0eb8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f58008c6640 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0eb8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f58008c6640 1 -- 172.21.15.110:0/565614056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599f0bf2620 con 0x5599f0eb39b0 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f57fb7fe640 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0eb8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f57db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.757+0000 7f58008c6640 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0ebf7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57f4010bf0 tx=0x7f57f4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ac00a6280d6974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f57db7fe640 1 -- 172.21.15.110:0/565614056 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f402c060 con 0x5599f0eb39b0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f57db7fe640 1 -- 172.21.15.110:0/565614056 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f57f4017030 con 0x5599f0eb39b0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 1 -- 172.21.15.110:0/565614056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5599f0e1f1f0 con 0x5599f0eb39b0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f57db7fe640 1 -- 172.21.15.110:0/565614056 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f57f401fbc0 con 0x5599f0eb39b0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 10 client.25879 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 1 -- 172.21.15.110:0/565614056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5599f0ec2c50 con 0x5599f0eb39b0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f57db7fe640 1 -- 172.21.15.110:0/565614056 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57f401fd60 con 0x5599f0eb39b0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f57db7fe640 1 -- 172.21.15.110:0/565614056 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f57f401c510 con 0x5599f0eb39b0 2024-04-25T00:15:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 10 client.25879 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 10 client.25879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 20 client.25879 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5599f0ec4390 0x5599f0ec6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 4 client.25879 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.758+0000 7f5805038740 1 -- 172.21.15.110:0/565614056 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599f0e1f1f0 con 0x5599f0ec4390 2024-04-25T00:15:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.759+0000 7f57fbfff640 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5599f0ec4390 0x5599f0ec6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.759+0000 7f57db7fe640 10 client.25879 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.759+0000 7f57fbfff640 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5599f0ec4390 0x5599f0ec6780 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb626de908b5d458 server_cookie=57b4844f39e44e0b in_seq=0 out_seq=0 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.760+0000 7f57db7fe640 1 -- 172.21.15.110:0/565614056 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599f0e1f1f0 con 0x5599f0ec4390 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.760+0000 7f57db7fe640 10 client.25879 handle_command_reply: tid=0 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.760+0000 7f5805038740 1 client.25879 shutdown 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.760+0000 7f5805038740 1 -- 172.21.15.110:0/565614056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 msgr2=0x5599f0ebf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.760+0000 7f5805038740 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0ebf7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57f4010bf0 tx=0x7f57f4004ab0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 1 -- 172.21.15.110:0/565614056 shutdown_connections 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5599f0ec4390 0x5599f0ec6780 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eb39b0 0x5599f0ebf7c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0eb8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 1 --2- 172.21.15.110:0/565614056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 0x5599f0eb8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 1 -- 172.21.15.110:0/565614056 >> 172.21.15.110:0/565614056 conn(0x5599f0ea94c0 msgr2=0x5599f0ea85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f57db7fe640 0 client.25879 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 1 -- 172.21.15.110:0/565614056 shutdown_connections 2024-04-25T00:15:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 1 -- 172.21.15.110:0/565614056 wait complete. 2024-04-25T00:15:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.761+0000 7f5805038740 20 client.25879 trim_cache size 0 max 16384 2024-04-25T00:15:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f5805038740 1 Processor -- start 2024-04-25T00:15:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f5805038740 1 -- start start 2024-04-25T00:15:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f5805038740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eade50 0x5599f0eac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f5805038740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0ebf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f5805038740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eb39b0 0x5599f0eac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f5805038740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1ad460 con 0x5599f0eae840 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f5805038740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1aad30 con 0x5599f0eb39b0 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f5805038740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1a8b80 con 0x5599f0eade50 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0ebf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0ebf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36598/0 (socket says 172.21.15.110:36598) 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fb7fe640 1 -- 172.21.15.110:0/952291012 learned_addr learned my addr 172.21.15.110:0/952291012 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eade50 0x5599f0eac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f58008c6640 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eb39b0 0x5599f0eac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fbfff640 1 -- 172.21.15.110:0/952291012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eb39b0 msgr2=0x5599f0eac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fbfff640 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eb39b0 0x5599f0eac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fbfff640 1 -- 172.21.15.110:0/952291012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 msgr2=0x5599f0ebf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fbfff640 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0ebf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fbfff640 1 -- 172.21.15.110:0/952291012 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599f0eaec20 con 0x5599f0eade50 2024-04-25T00:15:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.786+0000 7f57fb7fe640 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0ebf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.787+0000 7f57fbfff640 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eade50 0x5599f0eac240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0d7150 tx=0x7f57ec09c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d724f9a39c7240ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.787+0000 7f57e1ffb640 1 -- 172.21.15.110:0/952291012 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec09b580 con 0x5599f0eade50 2024-04-25T00:15:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.787+0000 7f57e1ffb640 1 -- 172.21.15.110:0/952291012 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ec09bc20 con 0x5599f0eade50 2024-04-25T00:15:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.787+0000 7f57e1ffb640 1 -- 172.21.15.110:0/952291012 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec09b760 con 0x5599f0eade50 2024-04-25T00:15:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.788+0000 7f5805038740 1 -- 172.21.15.110:0/952291012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eade50 msgr2=0x5599f0eac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.788+0000 7f5805038740 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eade50 0x5599f0eac240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0d7150 tx=0x7f57ec09c490 comp rx=0 tx=0).stop 2024-04-25T00:15:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.788+0000 7f5805038740 1 -- 172.21.15.110:0/952291012 shutdown_connections 2024-04-25T00:15:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.788+0000 7f5805038740 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eb39b0 0x5599f0eac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.788+0000 7f5805038740 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eae840 0x5599f0ebf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.788+0000 7f5805038740 1 --2- 172.21.15.110:0/952291012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eade50 0x5599f0eac240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.788+0000 7f5805038740 1 -- 172.21.15.110:0/952291012 >> 172.21.15.110:0/952291012 conn(0x5599f0ea94c0 msgr2=0x5599f0ea98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.788+0000 7f5805038740 1 -- 172.21.15.110:0/952291012 shutdown_connections 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 -- 172.21.15.110:0/952291012 wait complete. 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 -- start start 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 Processor -- start 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 0x5599f0ebd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eae840 0x5599f0ebd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eb39b0 0x5599f0ebe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1a91f0 con 0x5599f0eb39b0 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1aa660 con 0x5599f0eade50 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1a98c0 con 0x5599f0eae840 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f5805038740 10 client.0 _ll_register_callbacks cb 0x7f57f4009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eae840 0x5599f0ebd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eae840 0x5599f0ebd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:39240/0 (socket says 172.21.15.110:39240) 2024-04-25T00:15:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.789+0000 7f57fb7fe640 1 -- 172.21.15.110:0/1028806727 learned_addr learned my addr 172.21.15.110:0/1028806727 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f58008c6640 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eb39b0 0x5599f0ebe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57fbfff640 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 0x5599f0ebd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57fb7fe640 1 -- 172.21.15.110:0/1028806727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 msgr2=0x5599f0ebd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57fb7fe640 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 0x5599f0ebd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57fb7fe640 1 -- 172.21.15.110:0/1028806727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eb39b0 msgr2=0x5599f0ebe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57fb7fe640 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eb39b0 0x5599f0ebe8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57fb7fe640 1 -- 172.21.15.110:0/1028806727 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599f0ebee90 con 0x5599f0eae840 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57fb7fe640 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eae840 0x5599f0ebd890 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009230 tx=0x7f57f00317b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51994249033c6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f58008c6640 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eb39b0 0x5599f0ebe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57db7fe640 1 -- 172.21.15.110:0/1028806727 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f004a060 con 0x5599f0eae840 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f57db7fe640 1 -- 172.21.15.110:0/1028806727 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f57f0004450 con 0x5599f0eae840 2024-04-25T00:15:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.790+0000 7f5805038740 1 -- 172.21.15.110:0/1028806727 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ec09bc20 con 0x5599f0eae840 2024-04-25T00:15:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f57db7fe640 1 -- 172.21.15.110:0/1028806727 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f57f0035040 con 0x5599f0eae840 2024-04-25T00:15:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f5805038740 10 client.25885 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f5805038740 1 -- 172.21.15.110:0/1028806727 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5599f0f2ada0 con 0x5599f0eae840 2024-04-25T00:15:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f57db7fe640 1 -- 172.21.15.110:0/1028806727 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ec09bc20 con 0x5599f0eae840 2024-04-25T00:15:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f57db7fe640 1 -- 172.21.15.110:0/1028806727 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f57f003c750 con 0x5599f0eae840 2024-04-25T00:15:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f5805038740 10 client.25885 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f5805038740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f5805038740 20 client.25885 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f5805038740 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5599f0f2b800 0x5599f0f2dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f5805038740 4 client.25885 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:15:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f5805038740 1 -- 172.21.15.110:0/1028806727 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599f0e19540 con 0x5599f0f2b800 2024-04-25T00:15:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.791+0000 7f57fbfff640 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5599f0f2b800 0x5599f0f2dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.792+0000 7f57fbfff640 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5599f0f2b800 0x5599f0f2dc20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88b24b1ac02ce01 server_cookie=3c94eb04b7503fe3 in_seq=0 out_seq=0 2024-04-25T00:15:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.792+0000 7f57db7fe640 10 client.25885 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.817+0000 7f57db7fe640 1 -- 172.21.15.110:0/1028806727 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599f0e19540 con 0x5599f0f2b800 2024-04-25T00:15:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.817+0000 7f57db7fe640 10 client.25885 handle_command_reply: tid=0 2024-04-25T00:15:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.817+0000 7f5805038740 1 client.25885 shutdown 2024-04-25T00:15:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 -- 172.21.15.110:0/1028806727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eae840 msgr2=0x5599f0ebd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eae840 0x5599f0ebd890 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009230 tx=0x7f57f00317b0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 -- 172.21.15.110:0/1028806727 shutdown_connections 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5599f0f2b800 0x5599f0f2dc20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599f0eb39b0 0x5599f0ebe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f57db7fe640 0 client.25885 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5599f0eae840 0x5599f0ebd890 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 --2- 172.21.15.110:0/1028806727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599f0eade50 0x5599f0ebd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 -- 172.21.15.110:0/1028806727 >> 172.21.15.110:0/1028806727 conn(0x5599f0ea94c0 msgr2=0x5599f0ea7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 -- 172.21.15.110:0/1028806727 shutdown_connections 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.818+0000 7f5805038740 1 -- 172.21.15.110:0/1028806727 wait complete. 2024-04-25T00:15:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.819+0000 7f5805038740 20 client.25885 trim_cache size 0 max 16384 2024-04-25T00:15:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:15:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:15:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:15:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:15:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 -- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f57e40a2ce0 msgr2=0x7f57e40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f57e40a2ce0 0x7f57e40a51a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5599f0eaf0a0 tx=0x7f57f4009500 comp rx=0 tx=0).stop 2024-04-25T00:15:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 -- 172.21.15.110:0/41355521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc100690 msgr2=0x7f57fc06aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc100690 0x7f57fc06aae0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e9c0 tx=0x7f57ec00ee80 comp rx=0 tx=0).stop 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 -- 172.21.15.110:0/41355521 shutdown_connections 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f57e40a2ce0 0x7f57e40a51a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc102240 0x7f57fc1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57fc101880 0x7f57fc19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 --2- 172.21.15.110:0/41355521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc100690 0x7f57fc06aae0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.820+0000 7f5802329640 1 -- 172.21.15.110:0/41355521 >> 172.21.15.110:0/41355521 conn(0x7f57fc0fbc00 msgr2=0x7f57fc104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.821+0000 7f5802329640 1 -- 172.21.15.110:0/41355521 shutdown_connections 2024-04-25T00:15:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:32.821+0000 7f5802329640 1 -- 172.21.15.110:0/41355521 wait complete. 2024-04-25T00:15:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:33 smithi173 ceph-mon[32402]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.1 MiB/s wr, 1.05k op/s 2024-04-25T00:15:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:33 smithi188 ceph-mon[33285]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.1 MiB/s wr, 1.05k op/s 2024-04-25T00:15:34.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:33 smithi110 ceph-mon[29187]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.1 MiB/s wr, 1.05k op/s 2024-04-25T00:15:36.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:35 smithi173 ceph-mon[32402]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 MiB/s wr, 885 op/s 2024-04-25T00:15:36.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:35 smithi188 ceph-mon[33285]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 MiB/s wr, 885 op/s 2024-04-25T00:15:36.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:35 smithi110 ceph-mon[29187]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 MiB/s wr, 885 op/s 2024-04-25T00:15:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 750.0 s, 876.6 tps, lat 114.137 ms stddev 22.953 2024-04-25T00:15:37.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:36 smithi173 ceph-mon[32402]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.6 MiB/s wr, 765 op/s 2024-04-25T00:15:37.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:36 smithi188 ceph-mon[33285]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.6 MiB/s wr, 765 op/s 2024-04-25T00:15:37.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:36 smithi110 ceph-mon[29187]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.6 MiB/s wr, 765 op/s 2024-04-25T00:15:40.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:39 smithi173 ceph-mon[32402]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 1.07k op/s 2024-04-25T00:15:40.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:39 smithi188 ceph-mon[33285]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 1.07k op/s 2024-04-25T00:15:40.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:39 smithi110 ceph-mon[29187]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 1.07k op/s 2024-04-25T00:15:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 755.0 s, 912.0 tps, lat 109.425 ms stddev 14.864 2024-04-25T00:15:42.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:41 smithi188 ceph-mon[33285]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 897 op/s 2024-04-25T00:15:42.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:41 smithi173 ceph-mon[32402]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 897 op/s 2024-04-25T00:15:42.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:41 smithi110 ceph-mon[29187]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 897 op/s 2024-04-25T00:15:44.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:43 smithi173 ceph-mon[32402]: pgmap v761: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-04-25T00:15:44.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:43 smithi188 ceph-mon[33285]: pgmap v761: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-04-25T00:15:44.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:43 smithi110 ceph-mon[29187]: pgmap v761: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.4 MiB/s wr, 1.08k op/s 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.968+0000 7f045d1f9640 1 -- 172.21.15.110:0/1734673088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04581033e0 msgr2=0x7f0458109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.968+0000 7f045d1f9640 1 --2- 172.21.15.110:0/1734673088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04581033e0 0x7f0458109c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0450009c10 tx=0x7f045002a6f0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.968+0000 7f045d1f9640 1 -- 172.21.15.110:0/1734673088 shutdown_connections 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.968+0000 7f045d1f9640 1 --2- 172.21.15.110:0/1734673088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04581033e0 0x7f0458109c70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.968+0000 7f045d1f9640 1 --2- 172.21.15.110:0/1734673088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0458102a20 0x7f0458102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.968+0000 7f045d1f9640 1 --2- 172.21.15.110:0/1734673088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04580ffa50 0x7f04580ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.968+0000 7f045d1f9640 1 -- 172.21.15.110:0/1734673088 >> 172.21.15.110:0/1734673088 conn(0x7f04580fb3a0 msgr2=0x7f04580fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.968+0000 7f045d1f9640 1 -- 172.21.15.110:0/1734673088 shutdown_connections 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.969+0000 7f045d1f9640 1 -- 172.21.15.110:0/1734673088 wait complete. 2024-04-25T00:15:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.969+0000 7f045d1f9640 1 Processor -- start 2024-04-25T00:15:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045d1f9640 1 -- start start 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045d1f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04580ffa50 0x7f0458199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045d1f9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0458102a20 0x7f045819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045d1f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04581033e0 0x7f04581a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045d1f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0458101f00 con 0x7f04581033e0 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045d1f9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0458101d80 con 0x7f0458102a20 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045d1f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0458102080 con 0x7f04580ffa50 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04580ffa50 0x7f0458199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0458102a20 0x7f045819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04580ffa50 0x7f0458199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:51628/0 (socket says 172.21.15.110:51628) 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0458102a20 0x7f045819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47674/0 (socket says 172.21.15.110:47674) 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f0456d9d640 1 -- 172.21.15.110:0/11252837 learned_addr learned my addr 172.21.15.110:0/11252837 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.970+0000 7f045759e640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04581033e0 0x7f04581a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.971+0000 7f045659c640 1 -- 172.21.15.110:0/11252837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04580ffa50 msgr2=0x7f0458199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.971+0000 7f045659c640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04580ffa50 0x7f0458199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.971+0000 7f045659c640 1 -- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04581033e0 msgr2=0x7f04581a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.971+0000 7f045659c640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04581033e0 0x7f04581a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.971+0000 7f045659c640 1 -- 172.21.15.110:0/11252837 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04581a1710 con 0x7f0458102a20 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.971+0000 7f0456d9d640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04580ffa50 0x7f0458199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.971+0000 7f045759e640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04581033e0 0x7f04581a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.971+0000 7f045659c640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0458102a20 0x7f045819a030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f044c00e9f0 tx=0x7f044c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.972+0000 7f043ffff640 1 -- 172.21.15.110:0/11252837 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c00cd80 con 0x7f0458102a20 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.972+0000 7f043ffff640 1 -- 172.21.15.110:0/11252837 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f044c0035f0 con 0x7f0458102a20 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.972+0000 7f045d1f9640 1 -- 172.21.15.110:0/11252837 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04581a1a00 con 0x7f0458102a20 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.972+0000 7f045d1f9640 1 -- 172.21.15.110:0/11252837 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04581a1fa0 con 0x7f0458102a20 2024-04-25T00:15:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.972+0000 7f043ffff640 1 -- 172.21.15.110:0/11252837 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c010690 con 0x7f0458102a20 2024-04-25T00:15:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.975+0000 7f043ffff640 1 -- 172.21.15.110:0/11252837 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f044c0108f0 con 0x7f0458102a20 2024-04-25T00:15:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.975+0000 7f045ffe4740 1 Processor -- start 2024-04-25T00:15:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.975+0000 7f045ffe4740 1 -- start start 2024-04-25T00:15:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1d5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f045ffe4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f79c133cb0 con 0x55f79c1d0840 2024-04-25T00:15:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f045ffe4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f79c0dc5b0 con 0x55f79c1cfe50 2024-04-25T00:15:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f045ffe4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f79c12fc70 con 0x55f79c1d59b0 2024-04-25T00:15:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f043ffff640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f04400a2d30 0x7f04400a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1d5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57892/0 (socket says 172.21.15.110:57892) 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f0456d9d640 1 -- 172.21.15.110:0/2848726131 learned_addr learned my addr 172.21.15.110:0/2848726131 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f045759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.976+0000 7f045659c640 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f043ffff640 1 -- 172.21.15.110:0/11252837 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f044c0cf270 con 0x7f0458102a20 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f0456d9d640 1 -- 172.21.15.110:0/2848726131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 msgr2=0x55f79c1d7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f0456d9d640 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1d7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f0456d9d640 1 -- 172.21.15.110:0/2848726131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 msgr2=0x55f79c1d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f0456d9d640 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f0456d9d640 1 -- 172.21.15.110:0/2848726131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f79be48210 con 0x55f79c1d0840 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f045659c640 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1d7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f0456d9d640 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1d5340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0448009fd0 tx=0x7f0448011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b6346b39808da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f045759e640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f04400a2d30 0x7f04400a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.977+0000 7f043d7fa640 1 -- 172.21.15.110:0/2848726131 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044801e070 con 0x55f79c1d0840 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.978+0000 7f043d7fa640 1 -- 172.21.15.110:0/2848726131 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f044800faf0 con 0x55f79c1d0840 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.978+0000 7f043d7fa640 1 -- 172.21.15.110:0/2848726131 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448019c70 con 0x55f79c1d0840 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 -- 172.21.15.110:0/2848726131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 msgr2=0x55f79c1d5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1d5340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0448009fd0 tx=0x7f0448011a50 comp rx=0 tx=0).stop 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045759e640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f04400a2d30 0x7f04400a51f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0450034040 tx=0x7f045002a6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 -- 172.21.15.110:0/2848726131 shutdown_connections 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1d7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1d5340 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 --2- 172.21.15.110:0/2848726131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 -- 172.21.15.110:0/2848726131 >> 172.21.15.110:0/2848726131 conn(0x55f79c1cb4c0 msgr2=0x55f79c1cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 -- 172.21.15.110:0/2848726131 shutdown_connections 2024-04-25T00:15:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 -- 172.21.15.110:0/2848726131 wait complete. 2024-04-25T00:15:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 -- start start 2024-04-25T00:15:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.979+0000 7f045ffe4740 1 Processor -- start 2024-04-25T00:15:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1da1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d0840 0x55f79c1da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1e1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045ffe4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f79c1d8960 con 0x55f79c1cfe50 2024-04-25T00:15:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045ffe4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f79c0efb50 con 0x55f79c1d0840 2024-04-25T00:15:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045ffe4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f79c1d8ae0 con 0x55f79c1d59b0 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045ffe4740 10 client.0 _ll_register_callbacks cb 0x55f79c12aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1da1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d0840 0x55f79c1da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1da1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57894/0 (socket says 172.21.15.110:57894) 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f0456d9d640 1 -- 172.21.15.110:0/1558183212 learned_addr learned my addr 172.21.15.110:0/1558183212 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f045759e640 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f0456d9d640 1 -- 172.21.15.110:0/1558183212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 msgr2=0x55f79c1e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f0456d9d640 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f0456d9d640 1 -- 172.21.15.110:0/1558183212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d0840 msgr2=0x55f79c1da6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f0456d9d640 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d0840 0x55f79c1da6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.980+0000 7f0456d9d640 1 -- 172.21.15.110:0/1558183212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f79bf14620 con 0x55f79c1cfe50 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.981+0000 7f0456d9d640 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1da1b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0448000c00 tx=0x7f044800f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56294ee70b5262c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.981+0000 7f043aff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.981+0000 7f043aff5640 1 -- 172.21.15.110:0/1558183212 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448027020 con 0x55f79c1cfe50 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.981+0000 7f043aff5640 1 -- 172.21.15.110:0/1558183212 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0448004250 con 0x55f79c1cfe50 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.981+0000 7f045ffe4740 1 -- 172.21.15.110:0/1558183212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f044800faf0 con 0x55f79c1cfe50 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.981+0000 7f043aff5640 1 -- 172.21.15.110:0/1558183212 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f044801e070 con 0x55f79c1cfe50 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.981+0000 7f045ffe4740 10 client.16323 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.981+0000 7f045ffe4740 1 -- 172.21.15.110:0/1558183212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f79c1e4aa0 con 0x55f79c1cfe50 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.982+0000 7f043aff5640 1 -- 172.21.15.110:0/1558183212 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f044801f040 con 0x55f79c1cfe50 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.982+0000 7f043aff5640 1 -- 172.21.15.110:0/1558183212 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0448003870 con 0x55f79c1cfe50 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.982+0000 7f045ffe4740 10 client.16323 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.982+0000 7f045ffe4740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.982+0000 7f045ffe4740 20 client.16323 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.982+0000 7f045ffe4740 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f79c1e5ec0 0x55f79c1e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.982+0000 7f045ffe4740 4 client.16323 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.983+0000 7f045ffe4740 1 -- 172.21.15.110:0/1558183212 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f044800faf0 con 0x55f79c1e5ec0 2024-04-25T00:15:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.983+0000 7f045659c640 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f79c1e5ec0 0x55f79c1e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.983+0000 7f043aff5640 10 client.16323 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.983+0000 7f045659c640 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f79c1e5ec0 0x55f79c1e82b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef5f83ed1222312e server_cookie=9733df0cc0248766 in_seq=0 out_seq=0 2024-04-25T00:15:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f043aff5640 1 -- 172.21.15.110:0/1558183212 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f044800faf0 con 0x55f79c1e5ec0 2024-04-25T00:15:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f043aff5640 10 client.16323 handle_command_reply: tid=0 2024-04-25T00:15:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 client.16323 shutdown 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 -- 172.21.15.110:0/1558183212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 msgr2=0x55f79c1da1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1da1b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0448000c00 tx=0x7f044800f870 comp rx=0 tx=0).stop 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 -- 172.21.15.110:0/1558183212 shutdown_connections 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f79c1e5ec0 0x55f79c1e82b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d0840 0x55f79c1da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 --2- 172.21.15.110:0/1558183212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1da1b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.984+0000 7f045ffe4740 1 -- 172.21.15.110:0/1558183212 >> 172.21.15.110:0/1558183212 conn(0x55f79c1cb4c0 msgr2=0x55f79c1ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.985+0000 7f043aff5640 0 client.16323 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.985+0000 7f045ffe4740 1 -- 172.21.15.110:0/1558183212 shutdown_connections 2024-04-25T00:15:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.985+0000 7f045ffe4740 1 -- 172.21.15.110:0/1558183212 wait complete. 2024-04-25T00:15:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:45.985+0000 7f045ffe4740 20 client.16323 trim_cache size 0 max 16384 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045ffe4740 1 Processor -- start 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045ffe4740 1 -- start start 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1da490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045ffe4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04581ad560 con 0x55f79c1d0840 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045ffe4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04581aae30 con 0x55f79c1cfe50 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045ffe4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04581a8c80 con 0x55f79c1d59b0 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1da490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47702/0 (socket says 172.21.15.110:47702) 2024-04-25T00:15:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f0456d9d640 1 -- 172.21.15.110:0/2520557249 learned_addr learned my addr 172.21.15.110:0/2520557249 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.009+0000 7f045759e640 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f045759e640 1 -- 172.21.15.110:0/2520557249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 msgr2=0x55f79c1da490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f045759e640 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1da490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f045759e640 1 -- 172.21.15.110:0/2520557249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 msgr2=0x55f79c1e16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f045759e640 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1e16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f045759e640 1 -- 172.21.15.110:0/2520557249 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f79c1d8840 con 0x55f79c1d59b0 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f0456d9d640 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1da490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f045659c640 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f045759e640 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1ca440 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f045002ac10 tx=0x7f0450005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602c76878d834490 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f043cff9640 1 -- 172.21.15.110:0/2520557249 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450036040 con 0x55f79c1d59b0 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.010+0000 7f043cff9640 1 -- 172.21.15.110:0/2520557249 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0450045030 con 0x55f79c1d59b0 2024-04-25T00:15:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.011+0000 7f043cff9640 1 -- 172.21.15.110:0/2520557249 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450031a10 con 0x55f79c1d59b0 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 -- 172.21.15.110:0/2520557249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 msgr2=0x55f79c1ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1ca440 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f045002ac10 tx=0x7f0450005fd0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 -- 172.21.15.110:0/2520557249 shutdown_connections 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d59b0 0x55f79c1ca440 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1d0840 0x55f79c1e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 --2- 172.21.15.110:0/2520557249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1cfe50 0x55f79c1da490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 -- 172.21.15.110:0/2520557249 >> 172.21.15.110:0/2520557249 conn(0x55f79c1cb4c0 msgr2=0x55f79c1c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 -- 172.21.15.110:0/2520557249 shutdown_connections 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.012+0000 7f045ffe4740 1 -- 172.21.15.110:0/2520557249 wait complete. 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 1 -- start start 2024-04-25T00:15:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 1 Processor -- start 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d0840 0x55f79c1defb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d59b0 0x55f79c1dff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04581a92f0 con 0x55f79c1cfe50 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04581aa760 con 0x55f79c1d59b0 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04581a99c0 con 0x55f79c1d0840 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045ffe4740 10 client.0 _ll_register_callbacks cb 0x7f04480056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d59b0 0x55f79c1dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d59b0 0x55f79c1dff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47704/0 (socket says 172.21.15.110:47704) 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f0456d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1dea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57908/0 (socket says 172.21.15.110:57908) 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045759e640 1 -- 172.21.15.110:0/3619837893 learned_addr learned my addr 172.21.15.110:0/3619837893 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.013+0000 7f045659c640 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d0840 0x55f79c1defb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f045759e640 1 -- 172.21.15.110:0/3619837893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d0840 msgr2=0x55f79c1defb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f045759e640 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d0840 0x55f79c1defb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f045759e640 1 -- 172.21.15.110:0/3619837893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 msgr2=0x55f79c1dea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f045759e640 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1dea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f045759e640 1 -- 172.21.15.110:0/3619837893 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f79c1e0560 con 0x55f79c1d59b0 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f045759e640 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d59b0 0x55f79c1dff70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04500099a0 tx=0x7f0450005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f9002e7115a555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f043aff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f043aff5640 1 -- 172.21.15.110:0/3619837893 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450036030 con 0x55f79c1d59b0 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.014+0000 7f043aff5640 1 -- 172.21.15.110:0/3619837893 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0450004250 con 0x55f79c1d59b0 2024-04-25T00:15:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.015+0000 7f045ffe4740 1 -- 172.21.15.110:0/3619837893 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0450045030 con 0x55f79c1d59b0 2024-04-25T00:15:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.015+0000 7f043aff5640 1 -- 172.21.15.110:0/3619837893 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0450031440 con 0x55f79c1d59b0 2024-04-25T00:15:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.015+0000 7f045ffe4740 10 client.25919 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.015+0000 7f045ffe4740 1 -- 172.21.15.110:0/3619837893 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f79c1e1870 con 0x55f79c1d59b0 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.015+0000 7f043aff5640 1 -- 172.21.15.110:0/3619837893 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0450037040 con 0x55f79c1d59b0 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f043aff5640 1 -- 172.21.15.110:0/3619837893 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f045004d8d0 con 0x55f79c1d59b0 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f045ffe4740 10 client.25919 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f045ffe4740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f045ffe4740 20 client.25919 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f045ffe4740 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f79c24c910 0x55f79c24ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f045ffe4740 4 client.25919 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f045ffe4740 1 -- 172.21.15.110:0/3619837893 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0450045030 con 0x55f79c24c910 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f0456d9d640 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f79c24c910 0x55f79c24ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f0456d9d640 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f79c24c910 0x55f79c24ed00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3261654366f986fd server_cookie=dbe28a99fc6c508d in_seq=0 out_seq=0 2024-04-25T00:15:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.016+0000 7f043aff5640 10 client.25919 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f043aff5640 1 -- 172.21.15.110:0/3619837893 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0450045030 con 0x55f79c24c910 2024-04-25T00:15:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f043aff5640 10 client.25919 handle_command_reply: tid=0 2024-04-25T00:15:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 client.25919 shutdown 2024-04-25T00:15:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 -- 172.21.15.110:0/3619837893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d59b0 msgr2=0x55f79c1dff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d59b0 0x55f79c1dff70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04500099a0 tx=0x7f0450005e90 comp rx=0 tx=0).stop 2024-04-25T00:15:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 -- 172.21.15.110:0/3619837893 shutdown_connections 2024-04-25T00:15:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f79c24c910 0x55f79c24ed00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f79c1d59b0 0x55f79c1dff70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f043aff5640 0 client.25919 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f79c1d0840 0x55f79c1defb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 --2- 172.21.15.110:0/3619837893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f79c1cfe50 0x55f79c1dea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 -- 172.21.15.110:0/3619837893 >> 172.21.15.110:0/3619837893 conn(0x55f79c1cb4c0 msgr2=0x55f79c1ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 -- 172.21.15.110:0/3619837893 shutdown_connections 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.036+0000 7f045ffe4740 1 -- 172.21.15.110:0/3619837893 wait complete. 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.037+0000 7f045ffe4740 20 client.25919 trim_cache size 0 max 16384 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:15:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:15:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:15:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:15:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.037+0000 7f045d1f9640 1 -- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f04400a2d30 msgr2=0x7f04400a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.037+0000 7f045d1f9640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f04400a2d30 0x7f04400a51f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0450034040 tx=0x7f045002a6f0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.037+0000 7f045d1f9640 1 -- 172.21.15.110:0/11252837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0458102a20 msgr2=0x7f045819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.037+0000 7f045d1f9640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0458102a20 0x7f045819a030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f044c00e9f0 tx=0x7f044c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.038+0000 7f045d1f9640 1 -- 172.21.15.110:0/11252837 shutdown_connections 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.038+0000 7f045d1f9640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f04400a2d30 0x7f04400a51f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.038+0000 7f045d1f9640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04581033e0 0x7f04581a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.038+0000 7f045d1f9640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0458102a20 0x7f045819a030 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.038+0000 7f045d1f9640 1 --2- 172.21.15.110:0/11252837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04580ffa50 0x7f0458199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.038+0000 7f045d1f9640 1 -- 172.21.15.110:0/11252837 >> 172.21.15.110:0/11252837 conn(0x7f04580fb3a0 msgr2=0x7f0458105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.038+0000 7f045d1f9640 1 -- 172.21.15.110:0/11252837 shutdown_connections 2024-04-25T00:15:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:46.038+0000 7f045d1f9640 1 -- 172.21.15.110:0/11252837 wait complete. 2024-04-25T00:15:46.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:45 smithi173 ceph-mon[32402]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 948 op/s 2024-04-25T00:15:46.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:45 smithi188 ceph-mon[33285]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 948 op/s 2024-04-25T00:15:46.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:45 smithi110 ceph-mon[29187]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.4 MiB/s wr, 948 op/s 2024-04-25T00:15:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 760.0 s, 828.0 tps, lat 120.565 ms stddev 20.612 2024-04-25T00:15:47.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:46 smithi188 ceph-mon[33285]: pgmap v763: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 863 op/s 2024-04-25T00:15:47.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:46 smithi110 ceph-mon[29187]: pgmap v763: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 863 op/s 2024-04-25T00:15:47.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:46 smithi173 ceph-mon[32402]: pgmap v763: 97 pgs: 97 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 863 op/s 2024-04-25T00:15:50.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:49 smithi173 ceph-mon[32402]: pgmap v764: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-04-25T00:15:50.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:49 smithi188 ceph-mon[33285]: pgmap v764: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-04-25T00:15:50.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:49 smithi110 ceph-mon[29187]: pgmap v764: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.3 MiB/s wr, 1.08k op/s 2024-04-25T00:15:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 765.0 s, 810.2 tps, lat 123.427 ms stddev 19.820 2024-04-25T00:15:52.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:51 smithi173 ceph-mon[32402]: pgmap v765: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 932 op/s 2024-04-25T00:15:52.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:51 smithi188 ceph-mon[33285]: pgmap v765: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 932 op/s 2024-04-25T00:15:52.290 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:51 smithi110 ceph-mon[29187]: pgmap v765: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 932 op/s 2024-04-25T00:15:54.089 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:53 smithi110 ceph-mon[29187]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 1.12k op/s 2024-04-25T00:15:54.089 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:53 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:53 smithi173 ceph-mon[32402]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 1.12k op/s 2024-04-25T00:15:54.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:53 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:53 smithi188 ceph-mon[33285]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 1.12k op/s 2024-04-25T00:15:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:53 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:55.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:15:55.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:55.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:55.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:15:55.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:55.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:55.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:15:56.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:55 smithi173 ceph-mon[32402]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 MiB/s wr, 828 op/s 2024-04-25T00:15:56.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:55 smithi188 ceph-mon[33285]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 MiB/s wr, 828 op/s 2024-04-25T00:15:56.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:55 smithi110 ceph-mon[29187]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 MiB/s wr, 828 op/s 2024-04-25T00:15:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 770.0 s, 784.0 tps, lat 127.516 ms stddev 25.711 2024-04-25T00:15:57.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:56 smithi188 ceph-mon[33285]: pgmap v768: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.3 MiB/s wr, 729 op/s 2024-04-25T00:15:57.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:56 smithi110 ceph-mon[29187]: pgmap v768: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.3 MiB/s wr, 729 op/s 2024-04-25T00:15:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:56 smithi173 ceph-mon[32402]: pgmap v768: 97 pgs: 97 active+clean; 1.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.3 MiB/s wr, 729 op/s 2024-04-25T00:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 -- 172.21.15.110:0/2696103412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c075780 msgr2=0x7f4d7c073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2696103412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c075780 0x7f4d7c073bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009900 tx=0x7f4d7002a410 comp rx=0 tx=0).stop 2024-04-25T00:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 -- 172.21.15.110:0/2696103412 shutdown_connections 2024-04-25T00:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2696103412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d7c109660 0x7f4d7c10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2696103412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d7c0741a0 0x7f4d7c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2696103412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c075780 0x7f4d7c073bd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 -- 172.21.15.110:0/2696103412 >> 172.21.15.110:0/2696103412 conn(0x7f4d7c0fcd30 msgr2=0x7f4d7c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 -- 172.21.15.110:0/2696103412 shutdown_connections 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.169+0000 7f4d82c15640 1 -- 172.21.15.110:0/2696103412 wait complete. 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d82c15640 1 Processor -- start 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d82c15640 1 -- start start 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d82c15640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c0741a0 0x7f4d7c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d82c15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d7c075780 0x7f4d7c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d82c15640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d7c109660 0x7f4d7c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d82c15640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c10f500 con 0x7f4d7c109660 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d82c15640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c10f380 con 0x7f4d7c0741a0 2024-04-25T00:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d82c15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c10f680 con 0x7f4d7c075780 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d809b1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c0741a0 0x7f4d7c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d7bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d7c075780 0x7f4d7c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d809b1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c0741a0 0x7f4d7c199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:53800/0 (socket says 172.21.15.110:53800) 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d809b1640 1 -- 172.21.15.110:0/2733676087 learned_addr learned my addr 172.21.15.110:0/2733676087 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d811b2640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d7c109660 0x7f4d7c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d809b1640 1 -- 172.21.15.110:0/2733676087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d7c075780 msgr2=0x7f4d7c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d809b1640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d7c075780 0x7f4d7c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d809b1640 1 -- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d7c109660 msgr2=0x7f4d7c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d809b1640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d7c109660 0x7f4d7c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d809b1640 1 -- 172.21.15.110:0/2733676087 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c1a1700 con 0x7f4d7c0741a0 2024-04-25T00:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.170+0000 7f4d7bfff640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d7c075780 0x7f4d7c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.171+0000 7f4d809b1640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c0741a0 0x7f4d7c199ae0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4d700314e0 tx=0x7f4d70003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.171+0000 7f4d79ffb640 1 -- 172.21.15.110:0/2733676087 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7003f070 con 0x7f4d7c0741a0 2024-04-25T00:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.171+0000 7f4d79ffb640 1 -- 172.21.15.110:0/2733676087 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d70031c40 con 0x7f4d7c0741a0 2024-04-25T00:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.171+0000 7f4d79ffb640 1 -- 172.21.15.110:0/2733676087 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70035030 con 0x7f4d7c0741a0 2024-04-25T00:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.171+0000 7f4d82c15640 1 -- 172.21.15.110:0/2733676087 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c1a1990 con 0x7f4d7c0741a0 2024-04-25T00:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.171+0000 7f4d82c15640 1 -- 172.21.15.110:0/2733676087 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c1a1ed0 con 0x7f4d7c0741a0 2024-04-25T00:15:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d79ffb640 1 -- 172.21.15.110:0/2733676087 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4d70042070 con 0x7f4d7c0741a0 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d85a04740 1 Processor -- start 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d85a04740 1 -- start start 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96a98340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96a9ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d85a04740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ed969f6cb0 con 0x55ed96a989b0 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d85a04740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ed9699f5b0 con 0x55ed96a93840 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d85a04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ed969f2c70 con 0x55ed96a92e50 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d811b2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d7bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96a9ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d811b2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a93270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49906/0 (socket says 172.21.15.110:49906) 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d811b2640 1 -- 172.21.15.110:0/1258307858 learned_addr learned my addr 172.21.15.110:0/1258307858 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d809b1640 1 --2- 172.21.15.110:0/1258307858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96a98340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d79ffb640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d811b2640 1 -- 172.21.15.110:0/1258307858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 msgr2=0x55ed96a98340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d811b2640 1 --2- 172.21.15.110:0/1258307858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96a98340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d811b2640 1 -- 172.21.15.110:0/1258307858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 msgr2=0x55ed96a9ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.174+0000 7f4d811b2640 1 --2- 172.21.15.110:0/1258307858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96a9ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.175+0000 7f4d811b2640 1 -- 172.21.15.110:0/1258307858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9670b210 con 0x55ed96a92e50 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.175+0000 7f4d811b2640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.175+0000 7f4d79ffb640 1 -- 172.21.15.110:0/2733676087 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4d700ef6f0 con 0x7f4d7c0741a0 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.175+0000 7f4d811b2640 1 --2- 172.21.15.110:0/1258307858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a93270 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4d74011590 tx=0x7f4d74011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a722bce11a8ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.175+0000 7f4d811b2640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d7402b000 tx=0x7f4d74028000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.175+0000 7f4d66ffd640 1 -- 172.21.15.110:0/1258307858 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74004510 con 0x55ed96a92e50 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.175+0000 7f4d66ffd640 1 -- 172.21.15.110:0/1258307858 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d740046b0 con 0x55ed96a92e50 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.175+0000 7f4d66ffd640 1 -- 172.21.15.110:0/1258307858 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74025680 con 0x55ed96a92e50 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 -- 172.21.15.110:0/1258307858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 msgr2=0x55ed96a93270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 --2- 172.21.15.110:0/1258307858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a93270 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4d74011590 tx=0x7f4d74011a50 comp rx=0 tx=0).stop 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 -- 172.21.15.110:0/1258307858 shutdown_connections 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 --2- 172.21.15.110:0/1258307858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96a9ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 --2- 172.21.15.110:0/1258307858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96a98340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 --2- 172.21.15.110:0/1258307858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a93270 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 -- 172.21.15.110:0/1258307858 >> 172.21.15.110:0/1258307858 conn(0x55ed96a8e4c0 msgr2=0x55ed96a8e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 -- 172.21.15.110:0/1258307858 shutdown_connections 2024-04-25T00:15:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.176+0000 7f4d85a04740 1 -- 172.21.15.110:0/1258307858 wait complete. 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.177+0000 7f4d85a04740 1 -- start start 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.177+0000 7f4d85a04740 1 Processor -- start 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.177+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a92e50 0x55ed96a9d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.177+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a93840 0x55ed96a9d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a989b0 0x55ed96aa4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d85a04740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ed96a9b960 con 0x55ed96a92e50 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d85a04740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ed969b2b50 con 0x55ed96a989b0 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d85a04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ed96a9bae0 con 0x55ed96a93840 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d85a04740 10 client.0 _ll_register_callbacks cb 0x55ed969edca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d809b1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a92e50 0x55ed96a9d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d809b1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a92e50 0x55ed96a9d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33228/0 (socket says 172.21.15.110:33228) 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d809b1640 1 -- 172.21.15.110:0/4146791335 learned_addr learned my addr 172.21.15.110:0/4146791335 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d7bfff640 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a93840 0x55ed96a9d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d811b2640 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a989b0 0x55ed96aa4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d7bfff640 1 -- 172.21.15.110:0/4146791335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a989b0 msgr2=0x55ed96aa4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d7bfff640 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a989b0 0x55ed96aa4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d7bfff640 1 -- 172.21.15.110:0/4146791335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a92e50 msgr2=0x55ed96a9d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d7bfff640 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a92e50 0x55ed96a9d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d7bfff640 1 -- 172.21.15.110:0/4146791335 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed967d7620 con 0x55ed96a93840 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d7bfff640 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a93840 0x55ed96a9d840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c19aff0 tx=0x7f4d6800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a248000bc0ee65a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.178+0000 7f4d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.179+0000 7f4d5bfff640 1 -- 172.21.15.110:0/4146791335 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68012110 con 0x55ed96a93840 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.179+0000 7f4d5bfff640 1 -- 172.21.15.110:0/4146791335 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4d68004050 con 0x55ed96a93840 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.179+0000 7f4d85a04740 1 -- 172.21.15.110:0/4146791335 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d740046b0 con 0x55ed96a93840 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.179+0000 7f4d5bfff640 1 -- 172.21.15.110:0/4146791335 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4d680122b0 con 0x55ed96a93840 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.179+0000 7f4d85a04740 10 client.25915 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.179+0000 7f4d85a04740 1 -- 172.21.15.110:0/4146791335 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ed96aa7cc0 con 0x55ed96a93840 2024-04-25T00:15:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.179+0000 7f4d5bfff640 1 -- 172.21.15.110:0/4146791335 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d740046b0 con 0x55ed96a93840 2024-04-25T00:15:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.179+0000 7f4d5bfff640 1 -- 172.21.15.110:0/4146791335 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f4d68010430 con 0x55ed96a93840 2024-04-25T00:15:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d85a04740 10 client.25915 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d85a04740 10 client.25915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d85a04740 20 client.25915 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d85a04740 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55ed96aa97d0 0x55ed96aabc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d85a04740 4 client.25915 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d85a04740 1 -- 172.21.15.110:0/4146791335 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed96a041f0 con 0x55ed96aa97d0 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d809b1640 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55ed96aa97d0 0x55ed96aabc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d809b1640 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55ed96aa97d0 0x55ed96aabc70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7455d8df92b3e5 server_cookie=35f8c38d65faaa10 in_seq=0 out_seq=0 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.180+0000 7f4d5bfff640 10 client.25915 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d5bfff640 1 -- 172.21.15.110:0/4146791335 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed96a041f0 con 0x55ed96aa97d0 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d5bfff640 10 client.25915 handle_command_reply: tid=0 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 client.25915 shutdown 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 -- 172.21.15.110:0/4146791335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a93840 msgr2=0x55ed96a9d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a93840 0x55ed96a9d840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c19aff0 tx=0x7f4d6800f790 comp rx=0 tx=0).stop 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 -- 172.21.15.110:0/4146791335 shutdown_connections 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55ed96aa97d0 0x55ed96aabc70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a989b0 0x55ed96aa4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a93840 0x55ed96a9d840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 --2- 172.21.15.110:0/4146791335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a92e50 0x55ed96a9d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 -- 172.21.15.110:0/4146791335 >> 172.21.15.110:0/4146791335 conn(0x55ed96a8e4c0 msgr2=0x55ed96a8d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d5bfff640 0 client.25915 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.182+0000 7f4d85a04740 1 -- 172.21.15.110:0/4146791335 shutdown_connections 2024-04-25T00:15:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.183+0000 7f4d85a04740 1 -- 172.21.15.110:0/4146791335 wait complete. 2024-04-25T00:15:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.183+0000 7f4d85a04740 20 client.25915 trim_cache size 0 max 16384 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.203+0000 7f4d85a04740 1 Processor -- start 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.203+0000 7f4d85a04740 1 -- start start 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.203+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a94d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.203+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96a95250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.203+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d85a04740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1a9110 con 0x55ed96a989b0 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d85a04740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1a69e0 con 0x55ed96a93840 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d85a04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1a4830 con 0x55ed96a92e50 2024-04-25T00:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a94d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a94d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49932/0 (socket says 172.21.15.110:49932) 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 -- 172.21.15.110:0/910891992 learned_addr learned my addr 172.21.15.110:0/910891992 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d7bfff640 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96a95250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d811b2640 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 -- 172.21.15.110:0/910891992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 msgr2=0x55ed96a95250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96a95250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 -- 172.21.15.110:0/910891992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 msgr2=0x55ed96aa4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 -- 172.21.15.110:0/910891992 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed96a8f230 con 0x55ed96a92e50 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d809b1640 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a94d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d700b8a00 tx=0x7f4d700dc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e5c908284b7f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d667fc640 1 -- 172.21.15.110:0/910891992 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70049040 con 0x55ed96a92e50 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.204+0000 7f4d667fc640 1 -- 172.21.15.110:0/910891992 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d700c0330 con 0x55ed96a92e50 2024-04-25T00:15:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d667fc640 1 -- 172.21.15.110:0/910891992 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7003b460 con 0x55ed96a92e50 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 -- 172.21.15.110:0/910891992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 msgr2=0x55ed96a94d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a94d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d700b8a00 tx=0x7f4d700dc040 comp rx=0 tx=0).stop 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 -- 172.21.15.110:0/910891992 shutdown_connections 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96a95250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 --2- 172.21.15.110:0/910891992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96a94d10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 -- 172.21.15.110:0/910891992 >> 172.21.15.110:0/910891992 conn(0x55ed96a8e4c0 msgr2=0x55ed96a9d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 -- 172.21.15.110:0/910891992 shutdown_connections 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.205+0000 7f4d85a04740 1 -- 172.21.15.110:0/910891992 wait complete. 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 1 -- start start 2024-04-25T00:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 1 Processor -- start 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96aa19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96aa1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1a4ea0 con 0x55ed96a989b0 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1a6310 con 0x55ed96a93840 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1a5570 con 0x55ed96a92e50 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d85a04740 10 client.0 _ll_register_callbacks cb 0x7f4d68009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d811b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d7bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96aa1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d811b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33252/0 (socket says 172.21.15.110:33252) 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d811b2640 1 -- 172.21.15.110:0/781260161 learned_addr learned my addr 172.21.15.110:0/781260161 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.206+0000 7f4d809b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96aa19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d811b2640 1 -- 172.21.15.110:0/781260161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 msgr2=0x55ed96aa19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d811b2640 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96aa19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d811b2640 1 -- 172.21.15.110:0/781260161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 msgr2=0x55ed96aa1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d811b2640 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96aa1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d811b2640 1 -- 172.21.15.110:0/781260161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed96aa3510 con 0x55ed96a989b0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d7bfff640 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96aa1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d811b2640 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa2f20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4d74011590 tx=0x7f4d74009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb1d06aa90f1a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d5bfff640 1 -- 172.21.15.110:0/781260161 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400fea0 con 0x55ed96a989b0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d5bfff640 1 -- 172.21.15.110:0/781260161 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f4d74005030 con 0x55ed96a989b0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d85a04740 1 -- 172.21.15.110:0/781260161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d700c0330 con 0x55ed96a989b0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d5bfff640 1 -- 172.21.15.110:0/781260161 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4d7404a050 con 0x55ed96a989b0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d85a04740 10 client.16347 fetch_fsmap learned FSMap version 32 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.207+0000 7f4d85a04740 1 -- 172.21.15.110:0/781260161 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ed96aa4820 con 0x55ed96a989b0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d5bfff640 1 -- 172.21.15.110:0/781260161 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d700c0330 con 0x55ed96a989b0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d5bfff640 1 -- 172.21.15.110:0/781260161 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f4d74039040 con 0x55ed96a989b0 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d85a04740 10 client.16347 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:15:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d85a04740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:15:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d85a04740 20 client.16347 populate_metadata read hostname 'smithi110' 2024-04-25T00:15:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d85a04740 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55ed96b0ffd0 0x55ed96b123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d85a04740 4 client.16347 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:15:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d85a04740 1 -- 172.21.15.110:0/781260161 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed969fe540 con 0x55ed96b0ffd0 2024-04-25T00:15:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d809b1640 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55ed96b0ffd0 0x55ed96b123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d5bfff640 10 client.16347 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.208+0000 7f4d809b1640 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55ed96b0ffd0 0x55ed96b123c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc09981483d6a0 server_cookie=46e706fcb75c89d0 in_seq=0 out_seq=0 2024-04-25T00:15:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d5bfff640 1 -- 172.21.15.110:0/781260161 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed969fe540 con 0x55ed96b0ffd0 2024-04-25T00:15:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d5bfff640 10 client.16347 handle_command_reply: tid=0 2024-04-25T00:15:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 client.16347 shutdown 2024-04-25T00:15:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 -- 172.21.15.110:0/781260161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 msgr2=0x55ed96aa2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa2f20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4d74011590 tx=0x7f4d74009500 comp rx=0 tx=0).stop 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 -- 172.21.15.110:0/781260161 shutdown_connections 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55ed96b0ffd0 0x55ed96b123c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed96a989b0 0x55ed96aa2f20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d5bfff640 0 client.16347 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed96a93840 0x55ed96aa1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 --2- 172.21.15.110:0/781260161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ed96a92e50 0x55ed96aa19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 -- 172.21.15.110:0/781260161 >> 172.21.15.110:0/781260161 conn(0x55ed96a8e4c0 msgr2=0x55ed96a9d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 -- 172.21.15.110:0/781260161 shutdown_connections 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 1 -- 172.21.15.110:0/781260161 wait complete. 2024-04-25T00:15:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.227+0000 7f4d85a04740 20 client.16347 trim_cache size 0 max 16384 2024-04-25T00:15:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:15:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:15:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:15:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 -- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d6c0a2cb0 msgr2=0x7f4d6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d7402b000 tx=0x7f4d74028000 comp rx=0 tx=0).stop 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 -- 172.21.15.110:0/2733676087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c0741a0 msgr2=0x7f4d7c199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c0741a0 0x7f4d7c199ae0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4d700314e0 tx=0x7f4d70003c10 comp rx=0 tx=0).stop 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 -- 172.21.15.110:0/2733676087 shutdown_connections 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f4d6c0a2cb0 0x7f4d6c0a5170 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d7c109660 0x7f4d7c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d7c075780 0x7f4d7c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 --2- 172.21.15.110:0/2733676087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d7c0741a0 0x7f4d7c199ae0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 -- 172.21.15.110:0/2733676087 >> 172.21.15.110:0/2733676087 conn(0x7f4d7c0fcd30 msgr2=0x7f4d7c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 -- 172.21.15.110:0/2733676087 shutdown_connections 2024-04-25T00:15:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:15:59.228+0000 7f4d82c15640 1 -- 172.21.15.110:0/2733676087 wait complete. 2024-04-25T00:16:00.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:15:59 smithi173 ceph-mon[32402]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 MiB/s wr, 991 op/s 2024-04-25T00:16:00.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:15:59 smithi188 ceph-mon[33285]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 MiB/s wr, 991 op/s 2024-04-25T00:16:00.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:15:59 smithi110 ceph-mon[29187]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 MiB/s wr, 991 op/s 2024-04-25T00:16:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 775.0 s, 878.6 tps, lat 114.475 ms stddev 17.363 2024-04-25T00:16:02.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:01 smithi173 ceph-mon[32402]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.3 MiB/s wr, 822 op/s 2024-04-25T00:16:02.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:01 smithi188 ceph-mon[33285]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.3 MiB/s wr, 822 op/s 2024-04-25T00:16:02.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:01 smithi110 ceph-mon[29187]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.3 MiB/s wr, 822 op/s 2024-04-25T00:16:04.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:03 smithi173 ceph-mon[32402]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.3 MiB/s wr, 961 op/s 2024-04-25T00:16:04.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:03 smithi188 ceph-mon[33285]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.3 MiB/s wr, 961 op/s 2024-04-25T00:16:04.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:03 smithi110 ceph-mon[29187]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.3 MiB/s wr, 961 op/s 2024-04-25T00:16:05.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:04 smithi188 ceph-mon[33285]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.8 MiB/s wr, 783 op/s 2024-04-25T00:16:05.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:04 smithi110 ceph-mon[29187]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.8 MiB/s wr, 783 op/s 2024-04-25T00:16:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:04 smithi173 ceph-mon[32402]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.8 MiB/s wr, 783 op/s 2024-04-25T00:16:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 780.0 s, 855.0 tps, lat 116.158 ms stddev 19.829 2024-04-25T00:16:08.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:07 smithi173 ceph-mon[32402]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 730 op/s 2024-04-25T00:16:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:07 smithi188 ceph-mon[33285]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 730 op/s 2024-04-25T00:16:08.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:07 smithi110 ceph-mon[29187]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 730 op/s 2024-04-25T00:16:10.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:09 smithi173 ceph-mon[32402]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 1.04k op/s 2024-04-25T00:16:10.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:09 smithi188 ceph-mon[33285]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 1.04k op/s 2024-04-25T00:16:10.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:09 smithi110 ceph-mon[29187]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.1 MiB/s wr, 1.04k op/s 2024-04-25T00:16:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 785.0 s, 835.4 tps, lat 119.906 ms stddev 17.941 2024-04-25T00:16:12.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:11 smithi173 ceph-mon[32402]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 829 op/s 2024-04-25T00:16:12.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:11 smithi188 ceph-mon[33285]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 829 op/s 2024-04-25T00:16:12.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:11 smithi110 ceph-mon[29187]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 829 op/s 2024-04-25T00:16:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.353+0000 7f10eb33c640 1 -- 172.21.15.110:0/851150361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e406d360 msgr2=0x7f10e406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.353+0000 7f10eb33c640 1 --2- 172.21.15.110:0/851150361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e406d360 0x7f10e406d780 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009900 tx=0x7f10d402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.354+0000 7f10eb33c640 1 -- 172.21.15.110:0/851150361 shutdown_connections 2024-04-25T00:16:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.354+0000 7f10eb33c640 1 --2- 172.21.15.110:0/851150361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10e406ef40 0x7f10e406f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.354+0000 7f10eb33c640 1 --2- 172.21.15.110:0/851150361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10e406e5a0 0x7f10e406ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.354+0000 7f10eb33c640 1 --2- 172.21.15.110:0/851150361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e406d360 0x7f10e406d780 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.354+0000 7f10eb33c640 1 -- 172.21.15.110:0/851150361 >> 172.21.15.110:0/851150361 conn(0x7f10e406c050 msgr2=0x7f10e406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.354+0000 7f10eb33c640 1 -- 172.21.15.110:0/851150361 shutdown_connections 2024-04-25T00:16:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.354+0000 7f10eb33c640 1 -- 172.21.15.110:0/851150361 wait complete. 2024-04-25T00:16:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.355+0000 7f10eb33c640 1 Processor -- start 2024-04-25T00:16:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.355+0000 7f10eb33c640 1 -- start start 2024-04-25T00:16:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.355+0000 7f10eb33c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10e406d360 0x7f10e41a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.355+0000 7f10eb33c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10e406e5a0 0x7f10e41a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.355+0000 7f10eb33c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e406ef40 0x7f10e41adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10eb33c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10e411bf70 con 0x7f10e406e5a0 2024-04-25T00:16:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10eb33c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f10e411bdf0 con 0x7f10e406d360 2024-04-25T00:16:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10eb33c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10e411c0f0 con 0x7f10e406ef40 2024-04-25T00:16:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10ea33a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10e406d360 0x7f10e41a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10ea33a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10e406d360 0x7f10e41a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:58382/0 (socket says 172.21.15.110:58382) 2024-04-25T00:16:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10ea33a640 1 -- 172.21.15.110:0/3396405369 learned_addr learned my addr 172.21.15.110:0/3396405369 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10e9b39640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10e406e5a0 0x7f10e41a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10eab3b640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e406ef40 0x7f10e41adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10e9b39640 1 -- 172.21.15.110:0/3396405369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e406ef40 msgr2=0x7f10e41adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10e9b39640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e406ef40 0x7f10e41adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10e9b39640 1 -- 172.21.15.110:0/3396405369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10e406d360 msgr2=0x7f10e41a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10e9b39640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10e406d360 0x7f10e41a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10e9b39640 1 -- 172.21.15.110:0/3396405369 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e41ae250 con 0x7f10e406e5a0 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.356+0000 7f10ea33a640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10e406d360 0x7f10e41a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.357+0000 7f10e9b39640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10e406e5a0 0x7f10e41a6b70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f10e000c970 tx=0x7f10e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.357+0000 7f10db7fe640 1 -- 172.21.15.110:0/3396405369 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0007bb0 con 0x7f10e406e5a0 2024-04-25T00:16:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.357+0000 7f10db7fe640 1 -- 172.21.15.110:0/3396405369 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10e0007d50 con 0x7f10e406e5a0 2024-04-25T00:16:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.357+0000 7f10db7fe640 1 -- 172.21.15.110:0/3396405369 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0004e40 con 0x7f10e406e5a0 2024-04-25T00:16:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.357+0000 7f10eb33c640 1 -- 172.21.15.110:0/3396405369 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e41ae540 con 0x7f10e406e5a0 2024-04-25T00:16:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.357+0000 7f10eb33c640 1 -- 172.21.15.110:0/3396405369 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e41ae9e0 con 0x7f10e406e5a0 2024-04-25T00:16:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.361+0000 7f10ef388740 1 Processor -- start 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.362+0000 7f10db7fe640 1 -- 172.21.15.110:0/3396405369 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f10e0005020 con 0x7f10e406e5a0 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.363+0000 7f10ef388740 1 -- start start 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.363+0000 7f10ef388740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.363+0000 7f10ef388740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 0x5571f109c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.363+0000 7f10ef388740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f109cb20 0x5571f109efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.363+0000 7f10ef388740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5571f0ffacb0 con 0x5571f1096fc0 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.363+0000 7f10ef388740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5571f0fa35b0 con 0x5571f10979b0 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.363+0000 7f10ef388740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5571f0ff6c70 con 0x5571f109cb20 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10eab3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10eab3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50820/0 (socket says 172.21.15.110:50820) 2024-04-25T00:16:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10eab3b640 1 -- 172.21.15.110:0/4219708958 learned_addr learned my addr 172.21.15.110:0/4219708958 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10e9b39640 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f109cb20 0x5571f109efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10eab3b640 1 -- 172.21.15.110:0/4219708958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f109cb20 msgr2=0x5571f109efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10eab3b640 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f109cb20 0x5571f109efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10eab3b640 1 -- 172.21.15.110:0/4219708958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 msgr2=0x5571f109c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10eab3b640 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 0x5571f109c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10eab3b640 1 -- 172.21.15.110:0/4219708958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571f0d0f210 con 0x5571f1096fc0 2024-04-25T00:16:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.364+0000 7f10e9b39640 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f109cb20 0x5571f109efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.365+0000 7f10db7fe640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10d00a2d90 0x7f10d00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.365+0000 7f10eab3b640 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10973e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009fc0 tx=0x7f10dc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682c1b562ebe9bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.365+0000 7f10db7fe640 1 -- 172.21.15.110:0/3396405369 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f10e00cec40 con 0x7f10e406e5a0 2024-04-25T00:16:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.365+0000 7f10ea33a640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10d00a2d90 0x7f10d00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.365+0000 7f10d8bfe640 1 -- 172.21.15.110:0/4219708958 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc003b70 con 0x5571f1096fc0 2024-04-25T00:16:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.366+0000 7f10d8bfe640 1 -- 172.21.15.110:0/4219708958 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10dc003d10 con 0x5571f1096fc0 2024-04-25T00:16:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.366+0000 7f10d8bfe640 1 -- 172.21.15.110:0/4219708958 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc011950 con 0x5571f1096fc0 2024-04-25T00:16:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 -- 172.21.15.110:0/4219708958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 msgr2=0x5571f10973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10973e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009fc0 tx=0x7f10dc010860 comp rx=0 tx=0).stop 2024-04-25T00:16:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ea33a640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10d00a2d90 0x7f10d00a5250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5571f1097db0 tx=0x7f10d4005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 -- 172.21.15.110:0/4219708958 shutdown_connections 2024-04-25T00:16:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f109cb20 0x5571f109efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 0x5571f109c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 --2- 172.21.15.110:0/4219708958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10973e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 -- 172.21.15.110:0/4219708958 >> 172.21.15.110:0/4219708958 conn(0x5571f1092630 msgr2=0x5571f1092a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 -- 172.21.15.110:0/4219708958 shutdown_connections 2024-04-25T00:16:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 -- 172.21.15.110:0/4219708958 wait complete. 2024-04-25T00:16:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.367+0000 7f10ef388740 1 -- start start 2024-04-25T00:16:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ef388740 1 Processor -- start 2024-04-25T00:16:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ef388740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ef388740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f10979b0 0x5571f10a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ef388740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ef388740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5571f109fad0 con 0x5571f1096fc0 2024-04-25T00:16:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ef388740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5571f0fb6b50 con 0x5571f109cb20 2024-04-25T00:16:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ef388740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5571f109fc50 con 0x5571f10979b0 2024-04-25T00:16:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ef388740 10 client.0 _ll_register_callbacks cb 0x5571f0ff1ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ea33a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ea33a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50828/0 (socket says 172.21.15.110:50828) 2024-04-25T00:16:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10ea33a640 1 -- 172.21.15.110:0/3771918979 learned_addr learned my addr 172.21.15.110:0/3771918979 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10eab3b640 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.368+0000 7f10e9b39640 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f10979b0 0x5571f10a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10eab3b640 1 -- 172.21.15.110:0/3771918979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f10979b0 msgr2=0x5571f10a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10eab3b640 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f10979b0 0x5571f10a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10eab3b640 1 -- 172.21.15.110:0/3771918979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 msgr2=0x5571f10a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10eab3b640 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10eab3b640 1 -- 172.21.15.110:0/3771918979 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571f0ddb620 con 0x5571f109cb20 2024-04-25T00:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10eab3b640 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a8a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0095d0 tx=0x7f10dc004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa0f43a0f82b0eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10cdffb640 1 -- 172.21.15.110:0/3771918979 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc004e60 con 0x5571f109cb20 2024-04-25T00:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10ef388740 1 -- 172.21.15.110:0/3771918979 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10dc003d10 con 0x5571f109cb20 2024-04-25T00:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.369+0000 7f10cdffb640 1 -- 172.21.15.110:0/3771918979 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f10dc004410 con 0x5571f109cb20 2024-04-25T00:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10cdffb640 1 -- 172.21.15.110:0/3771918979 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f10dc011650 con 0x5571f109cb20 2024-04-25T00:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ef388740 10 client.25943 fetch_fsmap learned FSMap version 32 2024-04-25T00:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ef388740 1 -- 172.21.15.110:0/3771918979 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5571f10abd90 con 0x5571f109cb20 2024-04-25T00:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10cdffb640 1 -- 172.21.15.110:0/3771918979 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10dc003d10 con 0x5571f109cb20 2024-04-25T00:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10cdffb640 1 -- 172.21.15.110:0/3771918979 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f10dc0050d0 con 0x5571f109cb20 2024-04-25T00:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ef388740 10 client.25943 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ef388740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ef388740 20 client.25943 populate_metadata read hostname 'smithi110' 2024-04-25T00:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ef388740 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5571f10ad780 0x5571f10afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ef388740 4 client.25943 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ef388740 1 -- 172.21.15.110:0/3771918979 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571f10081f0 con 0x5571f10ad780 2024-04-25T00:16:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.370+0000 7f10ea33a640 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5571f10ad780 0x5571f10afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.371+0000 7f10cdffb640 10 client.25943 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.371+0000 7f10ea33a640 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5571f10ad780 0x5571f10afb90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c732a0a4fcda3b83 server_cookie=b8086eff033f5cac in_seq=0 out_seq=0 2024-04-25T00:16:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10cdffb640 1 -- 172.21.15.110:0/3771918979 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571f10081f0 con 0x5571f10ad780 2024-04-25T00:16:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10cdffb640 10 client.25943 handle_command_reply: tid=0 2024-04-25T00:16:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 client.25943 shutdown 2024-04-25T00:16:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 -- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 msgr2=0x5571f10a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a8a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0095d0 tx=0x7f10dc004c70 comp rx=0 tx=0).stop 2024-04-25T00:16:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 -- 172.21.15.110:0/3771918979 shutdown_connections 2024-04-25T00:16:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5571f10ad780 0x5571f10afb90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a8a20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f10979b0 0x5571f10a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 --2- 172.21.15.110:0/3771918979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f1096fc0 0x5571f10a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 -- 172.21.15.110:0/3771918979 >> 172.21.15.110:0/3771918979 conn(0x5571f1092630 msgr2=0x5571f1091830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10cdffb640 0 client.25943 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 -- 172.21.15.110:0/3771918979 shutdown_connections 2024-04-25T00:16:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.372+0000 7f10ef388740 1 -- 172.21.15.110:0/3771918979 wait complete. 2024-04-25T00:16:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.373+0000 7f10ef388740 20 client.25943 trim_cache size 0 max 16384 2024-04-25T00:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ef388740 1 Processor -- start 2024-04-25T00:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ef388740 1 -- start start 2024-04-25T00:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ef388740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f1098360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ef388740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 0x5571f10a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ef388740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f109cb20 0x5571f1098e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ef388740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41ba010 con 0x5571f109cb20 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ef388740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41b78e0 con 0x5571f10979b0 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ef388740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41b5730 con 0x5571f1096fc0 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10eab3b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f109cb20 0x5571f1098e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.394+0000 7f10ea33a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f1098360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10e9b39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 0x5571f10a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10ea33a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f1098360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:36046/0 (socket says 172.21.15.110:36046) 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10ea33a640 1 -- 172.21.15.110:0/2843097246 learned_addr learned my addr 172.21.15.110:0/2843097246 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10ea33a640 1 -- 172.21.15.110:0/2843097246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 msgr2=0x5571f10a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10ea33a640 1 --2- 172.21.15.110:0/2843097246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 0x5571f10a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10ea33a640 1 -- 172.21.15.110:0/2843097246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f109cb20 msgr2=0x5571f1098e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10ea33a640 1 --2- 172.21.15.110:0/2843097246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f109cb20 0x5571f1098e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10ea33a640 1 -- 172.21.15.110:0/2843097246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571f1096db0 con 0x5571f1096fc0 2024-04-25T00:16:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10e9b39640 1 --2- 172.21.15.110:0/2843097246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 0x5571f10a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10ea33a640 1 --2- 172.21.15.110:0/2843097246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f1098360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006010 tx=0x7f10d4040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f806c8112da751 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10cffff640 1 -- 172.21.15.110:0/2843097246 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4035040 con 0x5571f1096fc0 2024-04-25T00:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10cffff640 1 -- 172.21.15.110:0/2843097246 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10d4057070 con 0x5571f1096fc0 2024-04-25T00:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.395+0000 7f10cffff640 1 -- 172.21.15.110:0/2843097246 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4031910 con 0x5571f1096fc0 2024-04-25T00:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 -- 172.21.15.110:0/2843097246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 msgr2=0x5571f1098360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 --2- 172.21.15.110:0/2843097246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f1098360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006010 tx=0x7f10d4040000 comp rx=0 tx=0).stop 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 -- 172.21.15.110:0/2843097246 shutdown_connections 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 --2- 172.21.15.110:0/2843097246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f109cb20 0x5571f1098e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 --2- 172.21.15.110:0/2843097246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f10979b0 0x5571f10a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 --2- 172.21.15.110:0/2843097246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f1098360 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 -- 172.21.15.110:0/2843097246 >> 172.21.15.110:0/2843097246 conn(0x5571f1092630 msgr2=0x5571f1092a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 -- 172.21.15.110:0/2843097246 shutdown_connections 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.396+0000 7f10ef388740 1 -- 172.21.15.110:0/2843097246 wait complete. 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 1 -- start start 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 1 Processor -- start 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f10a8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f10979b0 0x5571f10a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41b5da0 con 0x5571f10979b0 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41b7210 con 0x5571f109cb20 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41b6470 con 0x5571f1096fc0 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10e9b39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f10979b0 0x5571f10a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10e9b39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f10979b0 0x5571f10a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50854/0 (socket says 172.21.15.110:50854) 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10e9b39640 1 -- 172.21.15.110:0/2709183372 learned_addr learned my addr 172.21.15.110:0/2709183372 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ea33a640 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f10a8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10eab3b640 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.397+0000 7f10ef388740 10 client.0 _ll_register_callbacks cb 0x7f10e00d24d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ea33a640 1 -- 172.21.15.110:0/2709183372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 msgr2=0x5571f10a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ea33a640 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ea33a640 1 -- 172.21.15.110:0/2709183372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f10979b0 msgr2=0x5571f10a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ea33a640 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f10979b0 0x5571f10a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ea33a640 1 -- 172.21.15.110:0/2709183372 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571f10a5b70 con 0x5571f1096fc0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10e9b39640 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f10979b0 0x5571f10a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ea33a640 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f10a8650 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f10d4005ab0 tx=0x7f10d40037f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0be41fc52c5b8ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10cdffb640 1 -- 172.21.15.110:0/2709183372 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4060060 con 0x5571f1096fc0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10cdffb640 1 -- 172.21.15.110:0/2709183372 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f10d4051310 con 0x5571f1096fc0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ef388740 1 -- 172.21.15.110:0/2709183372 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d4057070 con 0x5571f1096fc0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10cdffb640 1 -- 172.21.15.110:0/2709183372 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f10d4035040 con 0x5571f1096fc0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ef388740 10 client.25939 fetch_fsmap learned FSMap version 32 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.398+0000 7f10ef388740 1 -- 172.21.15.110:0/2709183372 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5571f1112c70 con 0x5571f1096fc0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10cdffb640 1 -- 172.21.15.110:0/2709183372 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10d4057070 con 0x5571f1096fc0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10cdffb640 1 -- 172.21.15.110:0/2709183372 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f10d403c440 con 0x5571f1096fc0 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10ef388740 10 client.25939 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10ef388740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10ef388740 20 client.25939 populate_metadata read hostname 'smithi110' 2024-04-25T00:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10ef388740 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5571f1113620 0x5571f1115a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10ef388740 4 client.25939 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10ef388740 1 -- 172.21.15.110:0/2709183372 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571f1002540 con 0x5571f1113620 2024-04-25T00:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.399+0000 7f10e9b39640 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5571f1113620 0x5571f1115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.400+0000 7f10e9b39640 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5571f1113620 0x5571f1115a40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1eaa97b42929803 server_cookie=69427a8e71f00ce8 in_seq=0 out_seq=0 2024-04-25T00:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.400+0000 7f10cdffb640 10 client.25939 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10cdffb640 1 -- 172.21.15.110:0/2709183372 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571f1002540 con 0x5571f1113620 2024-04-25T00:16:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10cdffb640 10 client.25939 handle_command_reply: tid=0 2024-04-25T00:16:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 client.25939 shutdown 2024-04-25T00:16:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 -- 172.21.15.110:0/2709183372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 msgr2=0x5571f10a8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f10a8650 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f10d4005ab0 tx=0x7f10d40037f0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 -- 172.21.15.110:0/2709183372 shutdown_connections 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5571f1113620 0x5571f1115a40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571f109cb20 0x5571f10a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10cdffb640 0 client.25939 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f10979b0 0x5571f10a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 --2- 172.21.15.110:0/2709183372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571f1096fc0 0x5571f10a8650 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 -- 172.21.15.110:0/2709183372 >> 172.21.15.110:0/2709183372 conn(0x5571f1092630 msgr2=0x5571f1098e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 -- 172.21.15.110:0/2709183372 shutdown_connections 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 1 -- 172.21.15.110:0/2709183372 wait complete. 2024-04-25T00:16:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.417+0000 7f10ef388740 20 client.25939 trim_cache size 0 max 16384 2024-04-25T00:16:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:16:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:16:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:16:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 -- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10d00a2d90 msgr2=0x7f10d00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10d00a2d90 0x7f10d00a5250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5571f1097db0 tx=0x7f10d4005bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 -- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10e406e5a0 msgr2=0x7f10e41a6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10e406e5a0 0x7f10e41a6b70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f10e000c970 tx=0x7f10e000ce30 comp rx=0 tx=0).stop 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 -- 172.21.15.110:0/3396405369 shutdown_connections 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f10d00a2d90 0x7f10d00a5250 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e406ef40 0x7f10e41adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10e406e5a0 0x7f10e41a6b70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 --2- 172.21.15.110:0/3396405369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10e406d360 0x7f10e41a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 -- 172.21.15.110:0/3396405369 >> 172.21.15.110:0/3396405369 conn(0x7f10e406c050 msgr2=0x7f10e410a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 -- 172.21.15.110:0/3396405369 shutdown_connections 2024-04-25T00:16:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:12.418+0000 7f10eb33c640 1 -- 172.21.15.110:0/3396405369 wait complete. 2024-04-25T00:16:13.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:12 smithi188 ceph-mon[33285]: pgmap v776: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 1.10k op/s 2024-04-25T00:16:13.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:12 smithi110 ceph-mon[29187]: pgmap v776: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 1.10k op/s 2024-04-25T00:16:13.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:12 smithi173 ceph-mon[32402]: pgmap v776: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 1.10k op/s 2024-04-25T00:16:16.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:15 smithi173 ceph-mon[32402]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.4 MiB/s wr, 859 op/s 2024-04-25T00:16:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:15 smithi188 ceph-mon[33285]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.4 MiB/s wr, 859 op/s 2024-04-25T00:16:16.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:15 smithi110 ceph-mon[29187]: pgmap v777: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.4 MiB/s wr, 859 op/s 2024-04-25T00:16:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 790.0 s, 885.0 tps, lat 112.922 ms stddev 14.737 2024-04-25T00:16:18.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:17 smithi173 ceph-mon[32402]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.9 MiB/s wr, 800 op/s 2024-04-25T00:16:18.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:17 smithi188 ceph-mon[33285]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.9 MiB/s wr, 800 op/s 2024-04-25T00:16:18.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:17 smithi110 ceph-mon[29187]: pgmap v778: 97 pgs: 97 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.9 MiB/s wr, 800 op/s 2024-04-25T00:16:19.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:18 smithi173 ceph-mon[32402]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-04-25T00:16:19.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:18 smithi188 ceph-mon[33285]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-04-25T00:16:19.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:18 smithi110 ceph-mon[29187]: pgmap v779: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-04-25T00:16:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 795.0 s, 857.0 tps, lat 116.893 ms stddev 19.457 2024-04-25T00:16:22.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:21 smithi173 ceph-mon[32402]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.7 MiB/s wr, 898 op/s 2024-04-25T00:16:22.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:21 smithi188 ceph-mon[33285]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.7 MiB/s wr, 898 op/s 2024-04-25T00:16:22.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:21 smithi110 ceph-mon[29187]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.7 MiB/s wr, 898 op/s 2024-04-25T00:16:24.089 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:23 smithi110 ceph-mon[29187]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 MiB/s wr, 1.14k op/s 2024-04-25T00:16:24.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:23 smithi173 ceph-mon[32402]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 MiB/s wr, 1.14k op/s 2024-04-25T00:16:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:23 smithi188 ceph-mon[33285]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.8 MiB/s wr, 1.14k op/s 2024-04-25T00:16:25.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:24 smithi188 ceph-mon[33285]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.6 MiB/s wr, 882 op/s 2024-04-25T00:16:25.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:24 smithi110 ceph-mon[29187]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.6 MiB/s wr, 882 op/s 2024-04-25T00:16:25.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:24 smithi173 ceph-mon[32402]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.6 MiB/s wr, 882 op/s 2024-04-25T00:16:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 -- 172.21.15.110:0/1335861791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58101890 msgr2=0x7f3f58101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1335861791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58101890 0x7f3f58101d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009900 tx=0x7f3f4c02a440 comp rx=0 tx=0).stop 2024-04-25T00:16:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 -- 172.21.15.110:0/1335861791 shutdown_connections 2024-04-25T00:16:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1335861791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f58102250 0x7f3f58068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1335861791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58101890 0x7f3f58101d10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1335861791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f581006a0 0x7f3f58100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 -- 172.21.15.110:0/1335861791 >> 172.21.15.110:0/1335861791 conn(0x7f3f580fbc10 msgr2=0x7f3f580fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 -- 172.21.15.110:0/1335861791 shutdown_connections 2024-04-25T00:16:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.552+0000 7f3f5d163640 1 -- 172.21.15.110:0/1335861791 wait complete. 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.553+0000 7f3f5d163640 1 Processor -- start 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.553+0000 7f3f5d163640 1 -- start start 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.553+0000 7f3f5d163640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f581006a0 0x7f3f58070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.553+0000 7f3f5d163640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58101890 0x7f3f58070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.553+0000 7f3f5d163640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58102250 0x7f3f581a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.553+0000 7f3f5d163640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5810f560 con 0x7f3f581006a0 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.553+0000 7f3f5d163640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5810f3e0 con 0x7f3f58102250 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.553+0000 7f3f5d163640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5810f6e0 con 0x7f3f58101890 2024-04-25T00:16:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f56d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f581006a0 0x7f3f58070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58102250 0x7f3f581a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f56d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f581006a0 0x7f3f58070150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50990/0 (socket says 172.21.15.110:50990) 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f56d9d640 1 -- 172.21.15.110:0/1470877253 learned_addr learned my addr 172.21.15.110:0/1470877253 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5659c640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58101890 0x7f3f58070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5759e640 1 -- 172.21.15.110:0/1470877253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58101890 msgr2=0x7f3f58070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5759e640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58101890 0x7f3f58070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5759e640 1 -- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f581006a0 msgr2=0x7f3f58070150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5759e640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f581006a0 0x7f3f58070150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5759e640 1 -- 172.21.15.110:0/1470877253 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f581aa180 con 0x7f3f58102250 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5659c640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58101890 0x7f3f58070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f5759e640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58102250 0x7f3f581a9c40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000be20 tx=0x7f3f5000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.554+0000 7f3f3ffff640 1 -- 172.21.15.110:0/1470877253 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50019070 con 0x7f3f58102250 2024-04-25T00:16:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.555+0000 7f3f5d163640 1 -- 172.21.15.110:0/1470877253 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f581aa350 con 0x7f3f58102250 2024-04-25T00:16:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.555+0000 7f3f3ffff640 1 -- 172.21.15.110:0/1470877253 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f500148f0 con 0x7f3f58102250 2024-04-25T00:16:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.555+0000 7f3f3ffff640 1 -- 172.21.15.110:0/1470877253 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f500127f0 con 0x7f3f58102250 2024-04-25T00:16:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.555+0000 7f3f5d163640 1 -- 172.21.15.110:0/1470877253 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f581aa890 con 0x7f3f58102250 2024-04-25T00:16:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.557+0000 7f3f5fe7b740 1 Processor -- start 2024-04-25T00:16:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.557+0000 7f3f3ffff640 1 -- 172.21.15.110:0/1470877253 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3f5001c070 con 0x7f3f58102250 2024-04-25T00:16:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.557+0000 7f3f5fe7b740 1 -- start start 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a03ede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618a0349cb0 con 0x5618a03e5e50 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5618a02f25b0 con 0x5618a03eb9b0 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618a0345c70 con 0x5618a03e6840 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f56d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03e6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50994/0 (socket says 172.21.15.110:50994) 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5659c640 1 -- 172.21.15.110:0/3551208024 learned_addr learned my addr 172.21.15.110:0/3551208024 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5659c640 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a03ede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f56d9d640 1 -- 172.21.15.110:0/3551208024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 msgr2=0x5618a03ede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a03ede40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f56d9d640 1 -- 172.21.15.110:0/3551208024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 msgr2=0x5618a03e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f56d9d640 1 -- 172.21.15.110:0/3551208024 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a005e210 con 0x5618a03e6840 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.558+0000 7f3f5659c640 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.559+0000 7f3f3ffff640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3f400a2c30 0x7f3f400a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.559+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03eb340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3f480058c0 tx=0x7f3f48011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec5697aba0a85ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.559+0000 7f3f5659c640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3f400a2c30 0x7f3f400a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.559+0000 7f3f3ffff640 1 -- 172.21.15.110:0/1470877253 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3f500ce840 con 0x7f3f58102250 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.560+0000 7f3f5659c640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3f400a2c30 0x7f3f400a50f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5618a03e70a0 tx=0x7f3f4c042040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.560+0000 7f3f3d7fa640 1 -- 172.21.15.110:0/3551208024 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48004220 con 0x5618a03e6840 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.560+0000 7f3f3d7fa640 1 -- 172.21.15.110:0/3551208024 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f480043c0 con 0x5618a03e6840 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.560+0000 7f3f3d7fa640 1 -- 172.21.15.110:0/3551208024 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48004cd0 con 0x5618a03e6840 2024-04-25T00:16:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.561+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/3551208024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 msgr2=0x5618a03eb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.561+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03eb340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3f480058c0 tx=0x7f3f48011810 comp rx=0 tx=0).stop 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.562+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/3551208024 shutdown_connections 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.562+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a03ede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.562+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03eb340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.562+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/3551208024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.562+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/3551208024 >> 172.21.15.110:0/3551208024 conn(0x5618a03e14c0 msgr2=0x5618a03e18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.562+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/3551208024 shutdown_connections 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.562+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/3551208024 wait complete. 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f5fe7b740 1 -- start start 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f5fe7b740 1 Processor -- start 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e5e50 0x5618a03f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e6840 0x5618a03f9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a0400090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618a03ee960 con 0x5618a03eb9b0 2024-04-25T00:16:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5618a0305b50 con 0x5618a03e6840 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f56d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e5e50 0x5618a03f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f56d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e5e50 0x5618a03f8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:40912/0 (socket says 172.21.15.110:40912) 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f56d9d640 1 -- 172.21.15.110:0/2350927568 learned_addr learned my addr 172.21.15.110:0/2350927568 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.563+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618a03eeae0 con 0x5618a03e5e50 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.564+0000 7f3f5659c640 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e6840 0x5618a03f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.564+0000 7f3f5fe7b740 10 client.0 _ll_register_callbacks cb 0x5618a0340ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.564+0000 7f3f5759e640 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a0400090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.564+0000 7f3f56d9d640 1 -- 172.21.15.110:0/2350927568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e6840 msgr2=0x5618a03f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.564+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e6840 0x5618a03f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.564+0000 7f3f56d9d640 1 -- 172.21.15.110:0/2350927568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 msgr2=0x5618a0400090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.564+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a0400090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.564+0000 7f3f56d9d640 1 -- 172.21.15.110:0/2350927568 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a012a620 con 0x5618a03e5e50 2024-04-25T00:16:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.565+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e5e50 0x5618a03f8b10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3f48000c00 tx=0x7f3f48004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5105cde5e40b9031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.565+0000 7f3f3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:16:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.565+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2350927568 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48027030 con 0x5618a03e5e50 2024-04-25T00:16:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.565+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2350927568 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3f48004fe0 con 0x5618a03e5e50 2024-04-25T00:16:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.565+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f480043c0 con 0x5618a03e5e50 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.566+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2350927568 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3f4801d040 con 0x5618a03e5e50 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.566+0000 7f3f5fe7b740 10 client.25951 fetch_fsmap learned FSMap version 32 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.566+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618a0403400 con 0x5618a03e5e50 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2350927568 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f480051e0 con 0x5618a03e5e50 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2350927568 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f3f4802e4c0 con 0x5618a03e5e50 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f5fe7b740 10 client.25951 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f5fe7b740 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f5fe7b740 20 client.25951 populate_metadata read hostname 'smithi110' 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618a0404e50 0x5618a04072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f5fe7b740 4 client.25951 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3f480043c0 con 0x5618a0404e50 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.567+0000 7f3f5659c640 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618a0404e50 0x5618a04072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.568+0000 7f3f3aff5640 10 client.25951 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.568+0000 7f3f5659c640 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618a0404e50 0x5618a04072b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e972a226a4205439 server_cookie=6b35181728ae71dc in_seq=0 out_seq=0 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2350927568 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3f480043c0 con 0x5618a0404e50 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f3aff5640 10 client.25951 handle_command_reply: tid=0 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 client.25951 shutdown 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e5e50 msgr2=0x5618a03f8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e5e50 0x5618a03f8b10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3f48000c00 tx=0x7f3f48004ef0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 shutdown_connections 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618a0404e50 0x5618a04072b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a0400090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e6840 0x5618a03f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f3aff5640 0 client.25951 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2350927568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e5e50 0x5618a03f8b10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 >> 172.21.15.110:0/2350927568 conn(0x5618a03e14c0 msgr2=0x5618a03e05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 shutdown_connections 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2350927568 wait complete. 2024-04-25T00:16:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.569+0000 7f3f5fe7b740 20 client.25951 trim_cache size 0 max 16384 2024-04-25T00:16:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.594+0000 7f3f5fe7b740 1 Processor -- start 2024-04-25T00:16:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.594+0000 7f3f5fe7b740 1 -- start start 2024-04-25T00:16:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e5e50 0x5618a03e18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a0400050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a03dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581b1b30 con 0x5618a03eb9b0 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581af400 con 0x5618a03e5e50 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581ad250 con 0x5618a03e6840 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a03dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a03dfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51022/0 (socket says 172.21.15.110:51022) 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5759e640 1 -- 172.21.15.110:0/2071495473 learned_addr learned my addr 172.21.15.110:0/2071495473 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e5e50 0x5618a03e18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f5659c640 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a0400050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f56d9d640 1 -- 172.21.15.110:0/2071495473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 msgr2=0x5618a0400050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a0400050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f56d9d640 1 -- 172.21.15.110:0/2071495473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 msgr2=0x5618a03dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a03dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f56d9d640 1 -- 172.21.15.110:0/2071495473 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618a03f9c90 con 0x5618a03e5e50 2024-04-25T00:16:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.595+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e5e50 0x5618a03e18a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3f48011850 tx=0x7f3f48009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f88553a81074d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.596+0000 7f3f3cff9640 1 -- 172.21.15.110:0/2071495473 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48027040 con 0x5618a03e5e50 2024-04-25T00:16:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.596+0000 7f3f3cff9640 1 -- 172.21.15.110:0/2071495473 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f4802e040 con 0x5618a03e5e50 2024-04-25T00:16:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.596+0000 7f3f3cff9640 1 -- 172.21.15.110:0/2071495473 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f480042a0 con 0x5618a03e5e50 2024-04-25T00:16:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2071495473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e5e50 msgr2=0x5618a03e18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e5e50 0x5618a03e18a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3f48011850 tx=0x7f3f48009530 comp rx=0 tx=0).stop 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2071495473 shutdown_connections 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03eb9b0 0x5618a03dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a0400050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2071495473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03e5e50 0x5618a03e18a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2071495473 >> 172.21.15.110:0/2071495473 conn(0x5618a03e14c0 msgr2=0x5618a03dc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2071495473 shutdown_connections 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2071495473 wait complete. 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 -- start start 2024-04-25T00:16:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.597+0000 7f3f5fe7b740 1 Processor -- start 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5fe7b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a0469420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581ad8c0 con 0x5618a03e5e50 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581aed30 con 0x5618a03eb9b0 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a0469420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a0469420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:51876/0 (socket says 172.21.15.110:51876) 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5759e640 1 -- 172.21.15.110:0/2077986624 learned_addr learned my addr 172.21.15.110:0/2077986624 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5659c640 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5fe7b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581adf90 con 0x5618a03e6840 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5fe7b740 10 client.0 _ll_register_callbacks cb 0x7f3f48005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5659c640 1 -- 172.21.15.110:0/2077986624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 msgr2=0x5618a0469420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5659c640 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a0469420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5659c640 1 -- 172.21.15.110:0/2077986624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 msgr2=0x5618a03fced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5659c640 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03fced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.598+0000 7f3f5659c640 1 -- 172.21.15.110:0/2077986624 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618a0469960 con 0x5618a03e6840 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.599+0000 7f3f5659c640 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03fd410 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c005c40 tx=0x7f3f4c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e2d6d5b5820fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.599+0000 7f3f3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.599+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2077986624 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0595f0 con 0x5618a03e6840 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.599+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2077986624 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f4802e040 con 0x5618a03e6840 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.599+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2077986624 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f3f4c059790 con 0x5618a03e6840 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.599+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2077986624 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3f4c069050 con 0x5618a03e6840 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.599+0000 7f3f5fe7b740 10 client.25957 fetch_fsmap learned FSMap version 32 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.599+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2077986624 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618a046ac80 con 0x5618a03e6840 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2077986624 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f4802e040 con 0x5618a03e6840 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2077986624 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f3f4c04a040 con 0x5618a03e6840 2024-04-25T00:16:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f5fe7b740 10 client.25957 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:16:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f5fe7b740 10 client.25957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f5fe7b740 20 client.25957 populate_metadata read hostname 'smithi110' 2024-04-25T00:16:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618a046b5e0 0x5618a046da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f5fe7b740 4 client.25957 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:16:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2077986624 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618a03571f0 con 0x5618a046b5e0 2024-04-25T00:16:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.600+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618a046b5e0 0x5618a046da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.601+0000 7f3f56d9d640 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618a046b5e0 0x5618a046da00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef2555033b6a53b server_cookie=412fd044cf668ac3 in_seq=0 out_seq=0 2024-04-25T00:16:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.601+0000 7f3f3aff5640 10 client.25957 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.631+0000 7f3f3aff5640 1 -- 172.21.15.110:0/2077986624 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618a03571f0 con 0x5618a046b5e0 2024-04-25T00:16:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.631+0000 7f3f3aff5640 10 client.25957 handle_command_reply: tid=0 2024-04-25T00:16:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.631+0000 7f3f5fe7b740 1 client.25957 shutdown 2024-04-25T00:16:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.635+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2077986624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 msgr2=0x5618a03fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.635+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03fd410 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c005c40 tx=0x7f3f4c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.636+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2077986624 shutdown_connections 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.636+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5618a046b5e0 0x5618a046da00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.636+0000 7f3f3aff5640 0 client.25957 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.636+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5618a03eb9b0 0x5618a0469420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.636+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618a03e6840 0x5618a03fd410 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.636+0000 7f3f5fe7b740 1 --2- 172.21.15.110:0/2077986624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618a03e5e50 0x5618a03fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.636+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2077986624 >> 172.21.15.110:0/2077986624 conn(0x5618a03e14c0 msgr2=0x5618a03e18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.636+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2077986624 shutdown_connections 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.637+0000 7f3f5fe7b740 1 -- 172.21.15.110:0/2077986624 wait complete. 2024-04-25T00:16:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.637+0000 7f3f5fe7b740 20 client.25957 trim_cache size 0 max 16384 2024-04-25T00:16:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:16:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:16:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:16:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:16:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 -- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3f400a2c30 msgr2=0x7f3f400a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3f400a2c30 0x7f3f400a50f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5618a03e70a0 tx=0x7f3f4c042040 comp rx=0 tx=0).stop 2024-04-25T00:16:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 -- 172.21.15.110:0/1470877253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58102250 msgr2=0x7f3f581a9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58102250 0x7f3f581a9c40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000be20 tx=0x7f3f5000bf10 comp rx=0 tx=0).stop 2024-04-25T00:16:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 -- 172.21.15.110:0/1470877253 shutdown_connections 2024-04-25T00:16:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3f400a2c30 0x7f3f400a50f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f58102250 0x7f3f581a9c40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f58101890 0x7f3f58070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 --2- 172.21.15.110:0/1470877253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f581006a0 0x7f3f58070150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 -- 172.21.15.110:0/1470877253 >> 172.21.15.110:0/1470877253 conn(0x7f3f580fbc10 msgr2=0x7f3f580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 -- 172.21.15.110:0/1470877253 shutdown_connections 2024-04-25T00:16:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:25.638+0000 7f3f5d163640 1 -- 172.21.15.110:0/1470877253 wait complete. 2024-04-25T00:16:26.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 800.0 s, 738.8 tps, lat 134.897 ms stddev 25.905 2024-04-25T00:16:28.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:27 smithi173 ceph-mon[32402]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.6 MiB/s wr, 877 op/s 2024-04-25T00:16:28.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:27 smithi188 ceph-mon[33285]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.6 MiB/s wr, 877 op/s 2024-04-25T00:16:28.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:27 smithi110 ceph-mon[29187]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.6 MiB/s wr, 877 op/s 2024-04-25T00:16:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:28 smithi188 ceph-mon[33285]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-04-25T00:16:29.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:28 smithi110 ceph-mon[29187]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-04-25T00:16:29.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:28 smithi173 ceph-mon[32402]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.23k op/s 2024-04-25T00:16:31.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:30 smithi188 ceph-mon[33285]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 981 op/s 2024-04-25T00:16:31.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:30 smithi110 ceph-mon[29187]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 981 op/s 2024-04-25T00:16:31.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:30 smithi173 ceph-mon[32402]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 981 op/s 2024-04-25T00:16:31.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 805.0 s, 609.2 tps, lat 163.343 ms stddev 29.297 2024-04-25T00:16:34.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:33 smithi173 ceph-mon[32402]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-04-25T00:16:34.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:33 smithi188 ceph-mon[33285]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-04-25T00:16:34.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:33 smithi110 ceph-mon[29187]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.30k op/s 2024-04-25T00:16:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:34 smithi188 ceph-mon[33285]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-04-25T00:16:35.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:34 smithi110 ceph-mon[29187]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-04-25T00:16:35.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:34 smithi173 ceph-mon[32402]: pgmap v787: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-04-25T00:16:36.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 810.0 s, 624.6 tps, lat 161.362 ms stddev 72.491 2024-04-25T00:16:37.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:36 smithi188 ceph-mon[33285]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-04-25T00:16:37.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:36 smithi110 ceph-mon[29187]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-04-25T00:16:37.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:36 smithi173 ceph-mon[32402]: pgmap v788: 97 pgs: 97 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-04-25T00:16:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 -- 172.21.15.110:0/3106374234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc494095b60 msgr2=0x7fc49409c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 --2- 172.21.15.110:0/3106374234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc494095b60 0x7fc49409c3f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc49c066eb0 tx=0x7fc49c098110 comp rx=0 tx=0).stop 2024-04-25T00:16:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 -- 172.21.15.110:0/3106374234 shutdown_connections 2024-04-25T00:16:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 --2- 172.21.15.110:0/3106374234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc494095b60 0x7fc49409c3f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 --2- 172.21.15.110:0/3106374234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4940951a0 0x7fc494095620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 --2- 172.21.15.110:0/3106374234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc494093fb0 0x7fc4940943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 -- 172.21.15.110:0/3106374234 >> 172.21.15.110:0/3106374234 conn(0x7fc49408f500 msgr2=0x7fc494091960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 -- 172.21.15.110:0/3106374234 shutdown_connections 2024-04-25T00:16:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.787+0000 7fc4a2472640 1 -- 172.21.15.110:0/3106374234 wait complete. 2024-04-25T00:16:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.788+0000 7fc4a2472640 1 Processor -- start 2024-04-25T00:16:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.788+0000 7fc4a2472640 1 -- start start 2024-04-25T00:16:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.788+0000 7fc4a2472640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc494093fb0 0x7fc49412d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.788+0000 7fc4a2472640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4940951a0 0x7fc49412d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.788+0000 7fc4a2472640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc494095b60 0x7fc494129de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.788+0000 7fc4a2472640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc49409f550 con 0x7fc494093fb0 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.788+0000 7fc4a2472640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc49409f3d0 con 0x7fc494095b60 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.788+0000 7fc4a2472640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc49409f6d0 con 0x7fc4940951a0 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a0c6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4940951a0 0x7fc49412d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a0c6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4940951a0 0x7fc49412d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:39050/0 (socket says 172.21.15.110:39050) 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a0c6f640 1 -- 172.21.15.110:0/776197298 learned_addr learned my addr 172.21.15.110:0/776197298 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a1c71640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc494095b60 0x7fc494129de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a0c6f640 1 -- 172.21.15.110:0/776197298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc494095b60 msgr2=0x7fc494129de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc494095b60 0x7fc494129de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a0c6f640 1 -- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc494093fb0 msgr2=0x7fc49412d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc494093fb0 0x7fc49412d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.789+0000 7fc4a0c6f640 1 -- 172.21.15.110:0/776197298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49412a6a0 con 0x7fc4940951a0 2024-04-25T00:16:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.790+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4940951a0 0x7fc49412d7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0078a0 tx=0x7fc48c007d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.790+0000 7fc4927fc640 1 -- 172.21.15.110:0/776197298 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c017020 con 0x7fc4940951a0 2024-04-25T00:16:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.790+0000 7fc4a2472640 1 -- 172.21.15.110:0/776197298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49413a590 con 0x7fc4940951a0 2024-04-25T00:16:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.790+0000 7fc4927fc640 1 -- 172.21.15.110:0/776197298 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc48c013400 con 0x7fc4940951a0 2024-04-25T00:16:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.790+0000 7fc4927fc640 1 -- 172.21.15.110:0/776197298 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0125c0 con 0x7fc4940951a0 2024-04-25T00:16:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.790+0000 7fc4a2472640 1 -- 172.21.15.110:0/776197298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49413a9a0 con 0x7fc4940951a0 2024-04-25T00:16:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.793+0000 7fc4927fc640 1 -- 172.21.15.110:0/776197298 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc48c01a020 con 0x7fc4940951a0 2024-04-25T00:16:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a63e6740 1 Processor -- start 2024-04-25T00:16:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a63e6740 1 -- start start 2024-04-25T00:16:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a4ff3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ff8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 0x5634a4ffb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a63e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5634a4f56cb0 con 0x5634a4ff8b90 2024-04-25T00:16:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a63e6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5634a4eff5b0 con 0x5634a4ff3a20 2024-04-25T00:16:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a63e6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5634a4f52c70 con 0x5634a4ff3030 2024-04-25T00:16:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1c71640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a4ff3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a0c6f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 0x5634a4ffb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1470640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ff8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1470640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ff8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:60258/0 (socket says 172.21.15.110:60258) 2024-04-25T00:16:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1470640 1 -- 172.21.15.110:0/1398287696 learned_addr learned my addr 172.21.15.110:0/1398287696 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1470640 1 -- 172.21.15.110:0/1398287696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 msgr2=0x5634a4ff3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1470640 1 --2- 172.21.15.110:0/1398287696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a4ff3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1470640 1 -- 172.21.15.110:0/1398287696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 msgr2=0x5634a4ffb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1470640 1 --2- 172.21.15.110:0/1398287696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 0x5634a4ffb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.794+0000 7fc4a1470640 1 -- 172.21.15.110:0/1398287696 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a4c6b210 con 0x5634a4ff3a20 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.795+0000 7fc4927fc640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4880a2c90 0x7fc4880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.795+0000 7fc4a1470640 1 --2- 172.21.15.110:0/1398287696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ff8520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc49800bb20 tx=0x7fc49800d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1393782c200c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.795+0000 7fc4927fc640 1 -- 172.21.15.110:0/776197298 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc48c0ceed0 con 0x7fc4940951a0 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.795+0000 7fc4877fe640 1 -- 172.21.15.110:0/1398287696 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498016020 con 0x5634a4ff3a20 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.795+0000 7fc4a1c71640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4880a2c90 0x7fc4880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.795+0000 7fc4877fe640 1 -- 172.21.15.110:0/1398287696 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc49800dd10 con 0x5634a4ff3a20 2024-04-25T00:16:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.795+0000 7fc4877fe640 1 -- 172.21.15.110:0/1398287696 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49800f7c0 con 0x5634a4ff3a20 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1398287696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 msgr2=0x5634a4ff8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1398287696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ff8520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc49800bb20 tx=0x7fc49800d2e0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a1c71640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4880a2c90 0x7fc4880a5150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0673f0 tx=0x7fc49c067a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1398287696 shutdown_connections 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1398287696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 0x5634a4ffb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1398287696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ff8520 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1398287696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a4ff3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1398287696 >> 172.21.15.110:0/1398287696 conn(0x5634a4fee6a0 msgr2=0x5634a4feeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1398287696 shutdown_connections 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.796+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1398287696 wait complete. 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.797+0000 7fc4a63e6740 1 -- start start 2024-04-25T00:16:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.797+0000 7fc4a63e6740 1 Processor -- start 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3030 0x5634a4ffd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ffdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a1470640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3030 0x5634a4ffd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a1470640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3030 0x5634a4ffd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44590/0 (socket says 172.21.15.110:44590) 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a1470640 1 -- 172.21.15.110:0/1172998583 learned_addr learned my addr 172.21.15.110:0/1172998583 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ffdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5004a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5634a4ffbb40 con 0x5634a4ff3030 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5634a4f12b50 con 0x5634a4ff3a20 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5634a4ffbcc0 con 0x5634a4ff8b90 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a63e6740 10 client.0 _ll_register_callbacks cb 0x5634a4f4dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.798+0000 7fc4a1c71640 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5004a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.799+0000 7fc4a1470640 1 -- 172.21.15.110:0/1172998583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 msgr2=0x5634a5004a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.799+0000 7fc4a1470640 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5004a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.799+0000 7fc4a1470640 1 -- 172.21.15.110:0/1172998583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 msgr2=0x5634a4ffdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.799+0000 7fc4a1470640 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ffdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.799+0000 7fc4a1470640 1 -- 172.21.15.110:0/1172998583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a4d37620 con 0x5634a4ff3030 2024-04-25T00:16:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.799+0000 7fc4a1470640 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3030 0x5634a4ffd5a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc498017000 tx=0x7fc498010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af193b7906bcdcc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.799+0000 7fc484ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:16:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.800+0000 7fc484ff9640 1 -- 172.21.15.110:0/1172998583 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498016040 con 0x5634a4ff3030 2024-04-25T00:16:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.800+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49800dd10 con 0x5634a4ff3030 2024-04-25T00:16:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.800+0000 7fc484ff9640 1 -- 172.21.15.110:0/1172998583 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc4980109c0 con 0x5634a4ff3030 2024-04-25T00:16:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.802+0000 7fc484ff9640 1 -- 172.21.15.110:0/1172998583 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc49802d070 con 0x5634a4ff3030 2024-04-25T00:16:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.802+0000 7fc4a63e6740 10 client.16389 fetch_fsmap learned FSMap version 32 2024-04-25T00:16:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.802+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634a5007db0 con 0x5634a4ff3030 2024-04-25T00:16:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.804+0000 7fc484ff9640 1 -- 172.21.15.110:0/1172998583 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc498010bc0 con 0x5634a4ff3030 2024-04-25T00:16:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.804+0000 7fc484ff9640 1 -- 172.21.15.110:0/1172998583 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc49801b420 con 0x5634a4ff3030 2024-04-25T00:16:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.804+0000 7fc4a63e6740 10 client.16389 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:16:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.804+0000 7fc4a63e6740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.804+0000 7fc4a63e6740 20 client.16389 populate_metadata read hostname 'smithi110' 2024-04-25T00:16:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.804+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5634a50097a0 0x5634a500bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.804+0000 7fc4a63e6740 4 client.16389 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.804+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc49800dd10 con 0x5634a50097a0 2024-04-25T00:16:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.805+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5634a50097a0 0x5634a500bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.805+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5634a50097a0 0x5634a500bbe0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8d06cbab412991 server_cookie=26361b48bb9dfc7 in_seq=0 out_seq=0 2024-04-25T00:16:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.805+0000 7fc484ff9640 10 client.16389 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.806+0000 7fc484ff9640 1 -- 172.21.15.110:0/1172998583 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc49800dd10 con 0x5634a50097a0 2024-04-25T00:16:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.806+0000 7fc484ff9640 10 client.16389 handle_command_reply: tid=0 2024-04-25T00:16:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.806+0000 7fc4a63e6740 1 client.16389 shutdown 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3030 msgr2=0x5634a4ffd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3030 0x5634a4ffd5a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc498017000 tx=0x7fc498010690 comp rx=0 tx=0).stop 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 shutdown_connections 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5634a50097a0 0x5634a500bbe0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5004a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a4ffdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc484ff9640 0 client.16389 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1172998583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3030 0x5634a4ffd5a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.807+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 >> 172.21.15.110:0/1172998583 conn(0x5634a4fee6a0 msgr2=0x5634a4fed8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.808+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 shutdown_connections 2024-04-25T00:16:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.813+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1172998583 wait complete. 2024-04-25T00:16:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.813+0000 7fc4a63e6740 20 client.16389 trim_cache size 0 max 16384 2024-04-25T00:16:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.840+0000 7fc4a63e6740 1 Processor -- start 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.840+0000 7fc4a63e6740 1 -- start start 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.840+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a500bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.840+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a5004a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1470640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a500bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1470640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a500bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:39070/0 (socket says 172.21.15.110:39070) 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1470640 1 -- 172.21.15.110:0/3716911485 learned_addr learned my addr 172.21.15.110:0/3716911485 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 0x5634a4ff1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a5004a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a63e6740 1 -- 172.21.15.110:0/3716911485 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc494146150 con 0x5634a4ff8b90 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a63e6740 1 -- 172.21.15.110:0/3716911485 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc494143a20 con 0x5634a4ff3a20 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a63e6740 1 -- 172.21.15.110:0/3716911485 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc494141870 con 0x5634a4ff3030 2024-04-25T00:16:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1c71640 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 0x5634a4ff1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1470640 1 -- 172.21.15.110:0/3716911485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 msgr2=0x5634a5004a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1470640 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a5004a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1470640 1 -- 172.21.15.110:0/3716911485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 msgr2=0x5634a4ff1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1470640 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 0x5634a4ff1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.841+0000 7fc4a1470640 1 -- 172.21.15.110:0/3716911485 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a4fe9940 con 0x5634a4ff3030 2024-04-25T00:16:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.842+0000 7fc4a1470640 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a500bbc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc49800d7e0 tx=0x7fc498004640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffa3d11b395f5ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.842+0000 7fc486ffd640 1 -- 172.21.15.110:0/3716911485 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498036020 con 0x5634a4ff3030 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.842+0000 7fc486ffd640 1 -- 172.21.15.110:0/3716911485 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc49801b620 con 0x5634a4ff3030 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.842+0000 7fc486ffd640 1 -- 172.21.15.110:0/3716911485 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498016070 con 0x5634a4ff3030 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 -- 172.21.15.110:0/3716911485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 msgr2=0x5634a500bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a500bbc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc49800d7e0 tx=0x7fc498004640 comp rx=0 tx=0).stop 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 -- 172.21.15.110:0/3716911485 shutdown_connections 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff8b90 0x5634a4ff1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a4ff3a20 0x5634a5004a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/3716911485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff3030 0x5634a500bbc0 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc49800d7e0 tx=0x7fc498004640 comp rx=0 tx=0).stop 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 -- 172.21.15.110:0/3716911485 >> 172.21.15.110:0/3716911485 conn(0x5634a4fee6a0 msgr2=0x5634a4ff1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 -- 172.21.15.110:0/3716911485 shutdown_connections 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.843+0000 7fc4a63e6740 1 -- 172.21.15.110:0/3716911485 wait complete. 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.844+0000 7fc4a63e6740 1 -- start start 2024-04-25T00:16:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.844+0000 7fc4a63e6740 1 Processor -- start 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3a20 0x5634a50017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5001d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a63e6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a50097a0 0x5634a507ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a1470640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3a20 0x5634a50017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a0c6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5001d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a1470640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3a20 0x5634a50017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44608/0 (socket says 172.21.15.110:44608) 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a1470640 1 -- 172.21.15.110:0/1846330377 learned_addr learned my addr 172.21.15.110:0/1846330377 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc494141ee0 con 0x5634a4ff3a20 2024-04-25T00:16:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc494143350 con 0x5634a50097a0 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4941425b0 con 0x5634a4ff8b90 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a63e6740 10 client.0 _ll_register_callbacks cb 0x7fc48c0d2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.845+0000 7fc4a1c71640 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a50097a0 0x5634a507ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.846+0000 7fc4a0c6f640 1 -- 172.21.15.110:0/1846330377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a50097a0 msgr2=0x5634a507ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.846+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a50097a0 0x5634a507ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.846+0000 7fc4a0c6f640 1 -- 172.21.15.110:0/1846330377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3a20 msgr2=0x5634a50017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.846+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3a20 0x5634a50017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.846+0000 7fc4a0c6f640 1 -- 172.21.15.110:0/1846330377 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a507f2a0 con 0x5634a4ff8b90 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.846+0000 7fc4a0c6f640 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5001d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc48c09b690 tx=0x7fc48c0d12a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf6d119f65c33af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.846+0000 7fc484ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc484ff9640 1 -- 172.21.15.110:0/1846330377 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0cce60 con 0x5634a4ff8b90 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc484ff9640 1 -- 172.21.15.110:0/1846330377 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc48c099a00 con 0x5634a4ff8b90 2024-04-25T00:16:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49801b620 con 0x5634a4ff8b90 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc484ff9640 1 -- 172.21.15.110:0/1846330377 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc48c09be00 con 0x5634a4ff8b90 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 10 client.25975 fetch_fsmap learned FSMap version 32 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634a50804e0 con 0x5634a4ff8b90 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc484ff9640 1 -- 172.21.15.110:0/1846330377 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc48c09b700 con 0x5634a4ff8b90 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc484ff9640 1 -- 172.21.15.110:0/1846330377 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc48c0135b0 con 0x5634a4ff8b90 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 10 client.25975 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 20 client.25975 populate_metadata read hostname 'smithi110' 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5634a5080df0 0x5634a5083210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 4 client.25975 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:16:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.847+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc49801b620 con 0x5634a5080df0 2024-04-25T00:16:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.848+0000 7fc4a1470640 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5634a5080df0 0x5634a5083210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.848+0000 7fc4a1470640 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5634a5080df0 0x5634a5083210 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a049d5281e0dcc server_cookie=6146503b1964566 in_seq=0 out_seq=0 2024-04-25T00:16:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.848+0000 7fc484ff9640 10 client.25975 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc484ff9640 1 -- 172.21.15.110:0/1846330377 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc49801b620 con 0x5634a5080df0 2024-04-25T00:16:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc484ff9640 10 client.25975 handle_command_reply: tid=0 2024-04-25T00:16:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 client.25975 shutdown 2024-04-25T00:16:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 msgr2=0x5634a5001d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5001d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc48c09b690 tx=0x7fc48c0d12a0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 shutdown_connections 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x5634a5080df0 0x5634a5083210 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc484ff9640 0 client.25975 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634a50097a0 0x5634a507ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634a4ff8b90 0x5634a5001d10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 --2- 172.21.15.110:0/1846330377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634a4ff3a20 0x5634a50017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.869+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 >> 172.21.15.110:0/1846330377 conn(0x5634a4fee6a0 msgr2=0x5634a4fed010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.870+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 shutdown_connections 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.870+0000 7fc4a63e6740 1 -- 172.21.15.110:0/1846330377 wait complete. 2024-04-25T00:16:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.870+0000 7fc4a63e6740 20 client.25975 trim_cache size 0 max 16384 2024-04-25T00:16:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:16:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:16:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:16:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:16:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 -- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4880a2c90 msgr2=0x7fc4880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4880a2c90 0x7fc4880a5150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0673f0 tx=0x7fc49c067a10 comp rx=0 tx=0).stop 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 -- 172.21.15.110:0/776197298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4940951a0 msgr2=0x7fc49412d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4940951a0 0x7fc49412d7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0078a0 tx=0x7fc48c007d60 comp rx=0 tx=0).stop 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 -- 172.21.15.110:0/776197298 shutdown_connections 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4880a2c90 0x7fc4880a5150 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc494095b60 0x7fc494129de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4940951a0 0x7fc49412d7b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 --2- 172.21.15.110:0/776197298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc494093fb0 0x7fc49412d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 -- 172.21.15.110:0/776197298 >> 172.21.15.110:0/776197298 conn(0x7fc49408f500 msgr2=0x7fc4940985c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 -- 172.21.15.110:0/776197298 shutdown_connections 2024-04-25T00:16:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:38.871+0000 7fc4a2472640 1 -- 172.21.15.110:0/776197298 wait complete. 2024-04-25T00:16:39.153 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:38 smithi173 ceph-mon[32402]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-04-25T00:16:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:38 smithi188 ceph-mon[33285]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-04-25T00:16:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:38 smithi110 ceph-mon[29187]: pgmap v789: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-04-25T00:16:41.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:40 smithi188 ceph-mon[33285]: pgmap v790: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-04-25T00:16:41.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:40 smithi110 ceph-mon[29187]: pgmap v790: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-04-25T00:16:41.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:40 smithi173 ceph-mon[32402]: pgmap v790: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-04-25T00:16:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 815.0 s, 591.8 tps, lat 167.729 ms stddev 76.153 2024-04-25T00:16:43.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:42 smithi188 ceph-mon[33285]: pgmap v791: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 1.39k op/s 2024-04-25T00:16:43.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:42 smithi110 ceph-mon[29187]: pgmap v791: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 1.39k op/s 2024-04-25T00:16:43.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:42 smithi173 ceph-mon[32402]: pgmap v791: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 1.39k op/s 2024-04-25T00:16:45.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:44 smithi188 ceph-mon[33285]: pgmap v792: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-04-25T00:16:45.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:44 smithi110 ceph-mon[29187]: pgmap v792: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-04-25T00:16:45.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:44 smithi173 ceph-mon[32402]: pgmap v792: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-04-25T00:16:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 820.0 s, 632.0 tps, lat 159.342 ms stddev 37.513 2024-04-25T00:16:47.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:46 smithi188 ceph-mon[33285]: pgmap v793: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:16:47.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:46 smithi110 ceph-mon[29187]: pgmap v793: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:16:47.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:46 smithi173 ceph-mon[32402]: pgmap v793: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:16:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:48 smithi188 ceph-mon[33285]: pgmap v794: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.31k op/s 2024-04-25T00:16:49.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:48 smithi110 ceph-mon[29187]: pgmap v794: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.31k op/s 2024-04-25T00:16:49.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:48 smithi173 ceph-mon[32402]: pgmap v794: 97 pgs: 97 active+clean; 1.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.31k op/s 2024-04-25T00:16:51.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:50 smithi188 ceph-mon[33285]: pgmap v795: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-04-25T00:16:51.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:50 smithi110 ceph-mon[29187]: pgmap v795: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-04-25T00:16:51.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:50 smithi173 ceph-mon[32402]: pgmap v795: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-04-25T00:16:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 825.0 s, 822.8 tps, lat 121.487 ms stddev 18.521 2024-04-25T00:16:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 -- 172.21.15.110:0/2305221886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c102270 msgr2=0x7f599c108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 --2- 172.21.15.110:0/2305221886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c102270 0x7f599c108b00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5994009ca0 tx=0x7f599402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 -- 172.21.15.110:0/2305221886 shutdown_connections 2024-04-25T00:16:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 --2- 172.21.15.110:0/2305221886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c102270 0x7f599c108b00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 --2- 172.21.15.110:0/2305221886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c1018b0 0x7f599c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 --2- 172.21.15.110:0/2305221886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f599c1006c0 0x7f599c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 -- 172.21.15.110:0/2305221886 >> 172.21.15.110:0/2305221886 conn(0x7f599c0fbc30 msgr2=0x7f599c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 -- 172.21.15.110:0/2305221886 shutdown_connections 2024-04-25T00:16:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 -- 172.21.15.110:0/2305221886 wait complete. 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.004+0000 7f59a3b09640 1 Processor -- start 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a3b09640 1 -- start start 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a3b09640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1006c0 0x7f599c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a3b09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c1018b0 0x7f599c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a3b09640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f599c102270 0x7f599c1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a3b09640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f599c10f5d0 con 0x7f599c1006c0 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a3b09640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f599c10f450 con 0x7f599c102270 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a3b09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f599c10f750 con 0x7f599c1018b0 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a18a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1006c0 0x7f599c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a10a4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c1018b0 0x7f599c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a20a6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f599c102270 0x7f599c1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a18a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1006c0 0x7f599c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51118/0 (socket says 172.21.15.110:51118) 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a10a4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c1018b0 0x7f599c19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:42606/0 (socket says 172.21.15.110:42606) 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a18a5640 1 -- 172.21.15.110:0/1364202279 learned_addr learned my addr 172.21.15.110:0/1364202279 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a18a5640 1 -- 172.21.15.110:0/1364202279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c1018b0 msgr2=0x7f599c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c1018b0 0x7f599c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a18a5640 1 -- 172.21.15.110:0/1364202279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f599c102270 msgr2=0x7f599c1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f599c102270 0x7f599c1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.005+0000 7f59a18a5640 1 -- 172.21.15.110:0/1364202279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599c1a17a0 con 0x7f599c1006c0 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.006+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1006c0 0x7f599c199b80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f599800b470 tx=0x7f599800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.006+0000 7f5992ffd640 1 -- 172.21.15.110:0/1364202279 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998015020 con 0x7f599c1006c0 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.006+0000 7f59a3b09640 1 -- 172.21.15.110:0/1364202279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599c1a1a90 con 0x7f599c1006c0 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.006+0000 7f5992ffd640 1 -- 172.21.15.110:0/1364202279 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5998011d40 con 0x7f599c1006c0 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.006+0000 7f5992ffd640 1 -- 172.21.15.110:0/1364202279 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599800f340 con 0x7f599c1006c0 2024-04-25T00:16:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.006+0000 7f59a3b09640 1 -- 172.21.15.110:0/1364202279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599c1a2030 con 0x7f599c1006c0 2024-04-25T00:16:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.009+0000 7f5992ffd640 1 -- 172.21.15.110:0/1364202279 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5998018020 con 0x7f599c1006c0 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a6813740 1 Processor -- start 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a6813740 1 -- start start 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a6813740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150ee50 0x55f63150f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a6813740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631514340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a6813740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 0x55f631516e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a6813740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f631472cb0 con 0x55f6315149b0 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a6813740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f63141b5b0 con 0x55f63150f840 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a6813740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f63146ec70 con 0x55f63150ee50 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a20a6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150ee50 0x55f63150f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a20a6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150ee50 0x55f63150f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:42616/0 (socket says 172.21.15.110:42616) 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a20a6640 1 -- 172.21.15.110:0/1820807268 learned_addr learned my addr 172.21.15.110:0/1820807268 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631514340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f59a10a4640 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 0x55f631516e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.010+0000 7f5992ffd640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59880a2cb0 0x7f59880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a18a5640 1 -- 172.21.15.110:0/1820807268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150ee50 msgr2=0x55f63150f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150ee50 0x55f63150f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a18a5640 1 -- 172.21.15.110:0/1820807268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 msgr2=0x55f631516e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 0x55f631516e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a18a5640 1 -- 172.21.15.110:0/1820807268 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f631187210 con 0x55f63150f840 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a20a6640 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150ee50 0x55f63150f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a20a6640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59880a2cb0 0x7f59880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f5992ffd640 1 -- 172.21.15.110:0/1364202279 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f59980ce9a0 con 0x7f599c1006c0 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631514340 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59980cc520 tx=0x7f5998021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c074fa0d7ddcc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f5987fff640 1 -- 172.21.15.110:0/1820807268 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998022540 con 0x55f63150f840 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f59a20a6640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59880a2cb0 0x7f59880a5170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55f6315100a0 tx=0x7f599403e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f5987fff640 1 -- 172.21.15.110:0/1820807268 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5998010570 con 0x55f63150f840 2024-04-25T00:16:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.011+0000 7f5987fff640 1 -- 172.21.15.110:0/1820807268 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998010ed0 con 0x55f63150f840 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.012+0000 7f59a6813740 1 -- 172.21.15.110:0/1820807268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 msgr2=0x55f631514340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.012+0000 7f59a6813740 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631514340 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59980cc520 tx=0x7f5998021000 comp rx=0 tx=0).stop 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.012+0000 7f59a6813740 1 -- 172.21.15.110:0/1820807268 shutdown_connections 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.012+0000 7f59a6813740 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 0x55f631516e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.012+0000 7f59a6813740 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631514340 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.013+0000 7f59a6813740 1 --2- 172.21.15.110:0/1820807268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150ee50 0x55f63150f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.013+0000 7f59a6813740 1 -- 172.21.15.110:0/1820807268 >> 172.21.15.110:0/1820807268 conn(0x55f63150a4c0 msgr2=0x55f63150a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.013+0000 7f59a6813740 1 -- 172.21.15.110:0/1820807268 shutdown_connections 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.013+0000 7f59a6813740 1 -- 172.21.15.110:0/1820807268 wait complete. 2024-04-25T00:16:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.013+0000 7f59a6813740 1 -- start start 2024-04-25T00:16:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a6813740 1 Processor -- start 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a6813740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f631519220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a6813740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150f840 0x55f631519760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a6813740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 0x55f6315207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a6813740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f631517900 con 0x55f6315149b0 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a6813740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f63142eb50 con 0x55f63150ee50 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a6813740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f631517a80 con 0x55f63150f840 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a18a5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f631519220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a10a4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150f840 0x55f631519760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a18a5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f631519220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47602/0 (socket says 172.21.15.110:47602) 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a18a5640 1 -- 172.21.15.110:0/1270774127 learned_addr learned my addr 172.21.15.110:0/1270774127 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a6813740 10 client.0 _ll_register_callbacks cb 0x55f631469ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.014+0000 7f59a20a6640 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 0x55f6315207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a18a5640 1 -- 172.21.15.110:0/1270774127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150f840 msgr2=0x55f631519760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150f840 0x55f631519760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a18a5640 1 -- 172.21.15.110:0/1270774127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 msgr2=0x55f6315207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 0x55f6315207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a18a5640 1 -- 172.21.15.110:0/1270774127 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f631253620 con 0x55f63150ee50 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a18a5640 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f631519220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5998097c00 tx=0x7f599809b310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57f8c609bb7b2f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59857fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59857fa640 1 -- 172.21.15.110:0/1270774127 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998022350 con 0x55f63150ee50 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59857fa640 1 -- 172.21.15.110:0/1270774127 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f59980978a0 con 0x55f63150ee50 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a6813740 1 -- 172.21.15.110:0/1270774127 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6314801f0 con 0x55f63150ee50 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59857fa640 1 -- 172.21.15.110:0/1270774127 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5998010ed0 con 0x55f63150ee50 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a6813740 10 client.25994 fetch_fsmap learned FSMap version 32 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.015+0000 7f59a6813740 1 -- 172.21.15.110:0/1270774127 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f631523b10 con 0x55f63150ee50 2024-04-25T00:16:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59857fa640 1 -- 172.21.15.110:0/1270774127 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f6314801f0 con 0x55f63150ee50 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59857fa640 1 -- 172.21.15.110:0/1270774127 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f599809f1e0 con 0x55f63150ee50 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59a6813740 10 client.25994 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59a6813740 10 client.25994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59a6813740 20 client.25994 populate_metadata read hostname 'smithi110' 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59a6813740 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f631524e60 0x55f631527250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59a6813740 4 client.25994 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59a6813740 1 -- 172.21.15.110:0/1270774127 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f63147a540 con 0x55f631524e60 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59a10a4640 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f631524e60 0x55f631527250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.016+0000 7f59a10a4640 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f631524e60 0x55f631527250 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d4df47c9cddd00 server_cookie=159c55843ef3dd9f in_seq=0 out_seq=0 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.017+0000 7f59857fa640 10 client.25994 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.017+0000 7f59857fa640 1 -- 172.21.15.110:0/1270774127 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f63147a540 con 0x55f631524e60 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.017+0000 7f59857fa640 10 client.25994 handle_command_reply: tid=0 2024-04-25T00:16:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.017+0000 7f59a6813740 1 client.25994 shutdown 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 -- 172.21.15.110:0/1270774127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 msgr2=0x55f631519220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f631519220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5998097c00 tx=0x7f599809b310 comp rx=0 tx=0).stop 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 -- 172.21.15.110:0/1270774127 shutdown_connections 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f631524e60 0x55f631527250 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59857fa640 0 client.25994 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6315149b0 0x55f6315207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f63150f840 0x55f631519760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 --2- 172.21.15.110:0/1270774127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f631519220 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 -- 172.21.15.110:0/1270774127 >> 172.21.15.110:0/1270774127 conn(0x55f63150a4c0 msgr2=0x55f6315095c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 -- 172.21.15.110:0/1270774127 shutdown_connections 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.018+0000 7f59a6813740 1 -- 172.21.15.110:0/1270774127 wait complete. 2024-04-25T00:16:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.019+0000 7f59a6813740 20 client.25994 trim_cache size 0 max 16384 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.042+0000 7f59a6813740 1 Processor -- start 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.042+0000 7f59a6813740 1 -- start start 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.042+0000 7f59a6813740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150ee50 0x55f631518e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.042+0000 7f59a6813740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631520730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.042+0000 7f59a6813740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631519370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.042+0000 7f59a6813740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1ad5c0 con 0x55f63150ee50 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.042+0000 7f59a6813740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1aae90 con 0x55f63150f840 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.042+0000 7f59a6813740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1a8ce0 con 0x55f6315149b0 2024-04-25T00:16:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a18a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150ee50 0x55f631518e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631520730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631520730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47610/0 (socket says 172.21.15.110:47610) 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a20a6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631519370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 -- 172.21.15.110:0/757664219 learned_addr learned my addr 172.21.15.110:0/757664219 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a18a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150ee50 0x55f631518e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51156/0 (socket says 172.21.15.110:51156) 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 -- 172.21.15.110:0/757664219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 msgr2=0x55f631519370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 --2- 172.21.15.110:0/757664219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631519370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 -- 172.21.15.110:0/757664219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150ee50 msgr2=0x55f631518e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 --2- 172.21.15.110:0/757664219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150ee50 0x55f631518e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 -- 172.21.15.110:0/757664219 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f631519f20 con 0x55f63150f840 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59a10a4640 1 --2- 172.21.15.110:0/757664219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631520730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f598c000c00 tx=0x7f598c0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e26ca6510c8d26c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59877fe640 1 -- 172.21.15.110:0/757664219 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c01f050 con 0x55f63150f840 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59877fe640 1 -- 172.21.15.110:0/757664219 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f598c01f6a0 con 0x55f63150f840 2024-04-25T00:16:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.043+0000 7f59877fe640 1 -- 172.21.15.110:0/757664219 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c010c20 con 0x55f63150f840 2024-04-25T00:16:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 -- 172.21.15.110:0/757664219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 msgr2=0x55f631520730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 --2- 172.21.15.110:0/757664219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631520730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f598c000c00 tx=0x7f598c0039c0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 -- 172.21.15.110:0/757664219 shutdown_connections 2024-04-25T00:16:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 --2- 172.21.15.110:0/757664219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631519370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 --2- 172.21.15.110:0/757664219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150f840 0x55f631520730 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 --2- 172.21.15.110:0/757664219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150ee50 0x55f631518e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 -- 172.21.15.110:0/757664219 >> 172.21.15.110:0/757664219 conn(0x55f63150a4c0 msgr2=0x55f6315100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 -- 172.21.15.110:0/757664219 shutdown_connections 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.044+0000 7f59a6813740 1 -- 172.21.15.110:0/757664219 wait complete. 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 1 -- start start 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 1 Processor -- start 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f63151d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150f840 0x55f63151db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631591a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1a9350 con 0x55f63150f840 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1aa7c0 con 0x55f63150ee50 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1a9a20 con 0x55f6315149b0 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a6813740 10 client.0 _ll_register_callbacks cb 0x7f598c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a18a5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f63151d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a18a5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f63151d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:47624/0 (socket says 172.21.15.110:47624) 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a18a5640 1 -- 172.21.15.110:0/3296637816 learned_addr learned my addr 172.21.15.110:0/3296637816 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:16:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a10a4640 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150f840 0x55f63151db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.045+0000 7f59a20a6640 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631591a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a18a5640 1 -- 172.21.15.110:0/3296637816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 msgr2=0x55f631591a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a18a5640 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631591a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a18a5640 1 -- 172.21.15.110:0/3296637816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150f840 msgr2=0x55f63151db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a18a5640 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150f840 0x55f63151db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a18a5640 1 -- 172.21.15.110:0/3296637816 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f631591f70 con 0x55f63150ee50 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a20a6640 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631591a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a18a5640 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f63151d640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f599809b1c0 tx=0x7f5998022730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe02fe26f52351a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59857fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59857fa640 1 -- 172.21.15.110:0/3296637816 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980b4230 con 0x55f63150ee50 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59857fa640 1 -- 172.21.15.110:0/3296637816 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f599809f150 con 0x55f63150ee50 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a6813740 1 -- 172.21.15.110:0/3296637816 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598c01f6a0 con 0x55f63150ee50 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59857fa640 1 -- 172.21.15.110:0/3296637816 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f599809f350 con 0x55f63150ee50 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a6813740 10 client.26000 fetch_fsmap learned FSMap version 32 2024-04-25T00:16:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.046+0000 7f59a6813740 1 -- 172.21.15.110:0/3296637816 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f631593280 con 0x55f63150ee50 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59857fa640 1 -- 172.21.15.110:0/3296637816 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f598c01f6a0 con 0x55f63150ee50 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59857fa640 1 -- 172.21.15.110:0/3296637816 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f59980228c0 con 0x55f63150ee50 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59a6813740 10 client.26000 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59a6813740 10 client.26000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59a6813740 20 client.26000 populate_metadata read hostname 'smithi110' 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59a6813740 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f631524e60 0x55f631595ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59a6813740 4 client.26000 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59a6813740 1 -- 172.21.15.110:0/3296637816 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6313f88e0 con 0x55f631524e60 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59a10a4640 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f631524e60 0x55f631595ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59857fa640 10 client.26000 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.047+0000 7f59a10a4640 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f631524e60 0x55f631595ae0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3a7154a8e34a037 server_cookie=6d611c8e4fb0e522 in_seq=0 out_seq=0 2024-04-25T00:16:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59857fa640 1 -- 172.21.15.110:0/3296637816 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6313f88e0 con 0x55f631524e60 2024-04-25T00:16:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59857fa640 10 client.26000 handle_command_reply: tid=0 2024-04-25T00:16:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 client.26000 shutdown 2024-04-25T00:16:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 -- 172.21.15.110:0/3296637816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 msgr2=0x55f63151d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f63151d640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f599809b1c0 tx=0x7f5998022730 comp rx=0 tx=0).stop 2024-04-25T00:16:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 -- 172.21.15.110:0/3296637816 shutdown_connections 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f631524e60 0x55f631595ae0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f6315149b0 0x55f631591a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f63150f840 0x55f63151db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59857fa640 0 client.26000 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 --2- 172.21.15.110:0/3296637816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f63150ee50 0x55f63151d640 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.077+0000 7f59a6813740 1 -- 172.21.15.110:0/3296637816 >> 172.21.15.110:0/3296637816 conn(0x55f63150a4c0 msgr2=0x55f631518e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a6813740 1 -- 172.21.15.110:0/3296637816 shutdown_connections 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a6813740 1 -- 172.21.15.110:0/3296637816 wait complete. 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a6813740 20 client.26000 trim_cache size 0 max 16384 2024-04-25T00:16:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:16:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:16:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:16:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:16:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a3b09640 1 -- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59880a2cb0 msgr2=0x7f59880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a3b09640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59880a2cb0 0x7f59880a5170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55f6315100a0 tx=0x7f599403e040 comp rx=0 tx=0).stop 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a3b09640 1 -- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1006c0 msgr2=0x7f599c199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a3b09640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1006c0 0x7f599c199b80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f599800b470 tx=0x7f599800b930 comp rx=0 tx=0).stop 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a3b09640 1 -- 172.21.15.110:0/1364202279 shutdown_connections 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.078+0000 7f59a3b09640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f59880a2cb0 0x7f59880a5170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.079+0000 7f59a3b09640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f599c102270 0x7f599c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.079+0000 7f59a3b09640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f599c1018b0 0x7f599c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.079+0000 7f59a3b09640 1 --2- 172.21.15.110:0/1364202279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1006c0 0x7f599c199b80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.079+0000 7f59a3b09640 1 -- 172.21.15.110:0/1364202279 >> 172.21.15.110:0/1364202279 conn(0x7f599c0fbc30 msgr2=0x7f599c104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.079+0000 7f59a3b09640 1 -- 172.21.15.110:0/1364202279 shutdown_connections 2024-04-25T00:16:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:16:52.079+0000 7f59a3b09640 1 -- 172.21.15.110:0/1364202279 wait complete. 2024-04-25T00:16:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:52 smithi188 ceph-mon[33285]: pgmap v796: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-04-25T00:16:53.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:52 smithi110 ceph-mon[29187]: pgmap v796: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-04-25T00:16:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:52 smithi173 ceph-mon[32402]: pgmap v796: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-04-25T00:16:54.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:54 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:54.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:54 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:54.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:54 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:55.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:55 smithi188 ceph-mon[33285]: pgmap v797: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 968 op/s 2024-04-25T00:16:55.538 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:55 smithi110 ceph-mon[29187]: pgmap v797: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 968 op/s 2024-04-25T00:16:55.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:55 smithi173 ceph-mon[32402]: pgmap v797: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 968 op/s 2024-04-25T00:16:56.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:56.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:56.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:16:56.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:56.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:56.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:16:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 830.0 s, 791.4 tps, lat 126.204 ms stddev 26.166 2024-04-25T00:16:56.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:56.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:56.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:16:57.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:57 smithi188 ceph-mon[33285]: pgmap v798: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.3 MiB/s wr, 888 op/s 2024-04-25T00:16:57.536 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:57 smithi110 ceph-mon[29187]: pgmap v798: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.3 MiB/s wr, 888 op/s 2024-04-25T00:16:57.602 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:57 smithi173 ceph-mon[32402]: pgmap v798: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.3 MiB/s wr, 888 op/s 2024-04-25T00:16:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:16:58 smithi188 ceph-mon[33285]: pgmap v799: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-04-25T00:16:59.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:16:58 smithi110 ceph-mon[29187]: pgmap v799: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-04-25T00:16:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:16:58 smithi173 ceph-mon[32402]: pgmap v799: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-04-25T00:17:01.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:00 smithi188 ceph-mon[33285]: pgmap v800: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.9 MiB/s wr, 852 op/s 2024-04-25T00:17:01.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:00 smithi110 ceph-mon[29187]: pgmap v800: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.9 MiB/s wr, 852 op/s 2024-04-25T00:17:01.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:00 smithi173 ceph-mon[32402]: pgmap v800: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.9 MiB/s wr, 852 op/s 2024-04-25T00:17:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 835.0 s, 839.4 tps, lat 119.336 ms stddev 17.518 2024-04-25T00:17:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:02 smithi188 ceph-mon[33285]: pgmap v801: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.8 MiB/s wr, 993 op/s 2024-04-25T00:17:03.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:02 smithi110 ceph-mon[29187]: pgmap v801: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.8 MiB/s wr, 993 op/s 2024-04-25T00:17:03.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:02 smithi173 ceph-mon[32402]: pgmap v801: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.8 MiB/s wr, 993 op/s 2024-04-25T00:17:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 -- 172.21.15.110:0/3608758216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b0103350 msgr2=0x7f05b0109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/3608758216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b0103350 0x7f05b0109be0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009be0 tx=0x7f05a802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 -- 172.21.15.110:0/3608758216 shutdown_connections 2024-04-25T00:17:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/3608758216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b0103350 0x7f05b0109be0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/3608758216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b0102990 0x7f05b0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/3608758216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b01017a0 0x7f05b0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 -- 172.21.15.110:0/3608758216 >> 172.21.15.110:0/3608758216 conn(0x7f05b00fcd10 msgr2=0x7f05b00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 -- 172.21.15.110:0/3608758216 shutdown_connections 2024-04-25T00:17:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.209+0000 7f05b4aa5640 1 -- 172.21.15.110:0/3608758216 wait complete. 2024-04-25T00:17:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.210+0000 7f05b4aa5640 1 Processor -- start 2024-04-25T00:17:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.210+0000 7f05b4aa5640 1 -- start start 2024-04-25T00:17:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.210+0000 7f05b4aa5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b01017a0 0x7f05b0197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.210+0000 7f05b4aa5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b0102990 0x7f05b0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05b4aa5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b0103350 0x7f05b019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05b4aa5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010d2b0 con 0x7f05b0103350 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05b4aa5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010d130 con 0x7f05b0102990 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05b4aa5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f05b010d430 con 0x7f05b01017a0 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b0102990 0x7f05b0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b0102990 0x7f05b0197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:60704/0 (socket says 172.21.15.110:60704) 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 -- 172.21.15.110:0/982236363 learned_addr learned my addr 172.21.15.110:0/982236363 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05ae59c640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b01017a0 0x7f05b0197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05aed9d640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b0103350 0x7f05b019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 -- 172.21.15.110:0/982236363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b01017a0 msgr2=0x7f05b0197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b01017a0 0x7f05b0197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 -- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b0103350 msgr2=0x7f05b019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b0103350 0x7f05b019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 -- 172.21.15.110:0/982236363 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b019f530 con 0x7f05b0102990 2024-04-25T00:17:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05ae59c640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b01017a0 0x7f05b0197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.211+0000 7f05a7fff640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b0102990 0x7f05b0197e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f059800e9c0 tx=0x7f059800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.212+0000 7f05a77fe640 1 -- 172.21.15.110:0/982236363 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059800cde0 con 0x7f05b0102990 2024-04-25T00:17:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.212+0000 7f05a77fe640 1 -- 172.21.15.110:0/982236363 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05980035f0 con 0x7f05b0102990 2024-04-25T00:17:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.212+0000 7f05a77fe640 1 -- 172.21.15.110:0/982236363 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598010690 con 0x7f05b0102990 2024-04-25T00:17:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.212+0000 7f05b4aa5640 1 -- 172.21.15.110:0/982236363 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b00739d0 con 0x7f05b0102990 2024-04-25T00:17:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.212+0000 7f05b4aa5640 1 -- 172.21.15.110:0/982236363 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b0073e70 con 0x7f05b0102990 2024-04-25T00:17:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.215+0000 7f05a77fe640 1 -- 172.21.15.110:0/982236363 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f05980037a0 con 0x7f05b0102990 2024-04-25T00:17:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.215+0000 7f05b77b3740 1 Processor -- start 2024-04-25T00:17:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.215+0000 7f05b77b3740 1 -- start start 2024-04-25T00:17:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.215+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fa7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63faee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05b77b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558d63f0acb0 con 0x558d63fa7840 2024-04-25T00:17:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05b77b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558d63eb35b0 con 0x558d63fac9b0 2024-04-25T00:17:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05b77b3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558d63f06c70 con 0x558d63fa6e50 2024-04-25T00:17:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63faee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05aed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fa7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05aed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fa7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:51188/0 (socket says 172.21.15.110:51188) 2024-04-25T00:17:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63faee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:60718/0 (socket says 172.21.15.110:60718) 2024-04-25T00:17:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05ae59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05aed9d640 1 -- 172.21.15.110:0/3084301008 learned_addr learned my addr 172.21.15.110:0/3084301008 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a7fff640 1 -- 172.21.15.110:0/3084301008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 msgr2=0x558d63fa7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a7fff640 1 --2- 172.21.15.110:0/3084301008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fa7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a7fff640 1 -- 172.21.15.110:0/3084301008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 msgr2=0x558d63fac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a7fff640 1 --2- 172.21.15.110:0/3084301008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a7fff640 1 -- 172.21.15.110:0/3084301008 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d63c1f210 con 0x558d63fac9b0 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a77fe640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05900a7140 0x7f05900a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.216+0000 7f05a7fff640 1 --2- 172.21.15.110:0/3084301008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63faee40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f05980ccde0 tx=0x7f05980d7780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954e88fe58d039c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.217+0000 7f05aed9d640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05900a7140 0x7f05900a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.217+0000 7f05a4ff9640 1 -- 172.21.15.110:0/3084301008 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05980e2070 con 0x558d63fac9b0 2024-04-25T00:17:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.217+0000 7f05a4ff9640 1 -- 172.21.15.110:0/3084301008 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05980d14e0 con 0x558d63fac9b0 2024-04-25T00:17:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.217+0000 7f05a4ff9640 1 -- 172.21.15.110:0/3084301008 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05980dd9d0 con 0x558d63fac9b0 2024-04-25T00:17:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.217+0000 7f05a77fe640 1 -- 172.21.15.110:0/982236363 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f05980cf370 con 0x7f05b0102990 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 -- 172.21.15.110:0/3084301008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 msgr2=0x558d63faee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05aed9d640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05900a7140 0x7f05900a9600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x558d63fa80a0 tx=0x7f05a8030940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 --2- 172.21.15.110:0/3084301008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63faee40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f05980ccde0 tx=0x7f05980d7780 comp rx=0 tx=0).stop 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 -- 172.21.15.110:0/3084301008 shutdown_connections 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 --2- 172.21.15.110:0/3084301008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63faee40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 --2- 172.21.15.110:0/3084301008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 --2- 172.21.15.110:0/3084301008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fa7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 -- 172.21.15.110:0/3084301008 >> 172.21.15.110:0/3084301008 conn(0x558d63fa24c0 msgr2=0x558d63fa28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 -- 172.21.15.110:0/3084301008 shutdown_connections 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.218+0000 7f05b77b3740 1 -- 172.21.15.110:0/3084301008 wait complete. 2024-04-25T00:17:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 1 -- start start 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 1 Processor -- start 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fb9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa7840 0x558d63fba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fac9b0 0x558d63fc1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558d63faf960 con 0x558d63fac9b0 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558d63ec6b50 con 0x558d63fa7840 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558d63fafae0 con 0x558d63fa6e50 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05b77b3740 10 client.0 _ll_register_callbacks cb 0x558d63f01ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05ae59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fb9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05a7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa7840 0x558d63fba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05ae59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fb9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:51192/0 (socket says 172.21.15.110:51192) 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05ae59c640 1 -- 172.21.15.110:0/1678101989 learned_addr learned my addr 172.21.15.110:0/1678101989 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05aed9d640 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fac9b0 0x558d63fc1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05a7fff640 1 -- 172.21.15.110:0/1678101989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 msgr2=0x558d63fb9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05a7fff640 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fb9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.219+0000 7f05a7fff640 1 -- 172.21.15.110:0/1678101989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fac9b0 msgr2=0x558d63fc1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f05a7fff640 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fac9b0 0x558d63fc1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f05a7fff640 1 -- 172.21.15.110:0/1678101989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d63ceb620 con 0x558d63fa7840 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f05aed9d640 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fac9b0 0x558d63fc1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f05a7fff640 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa7840 0x558d63fba050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f059809b8a0 tx=0x7f0598003d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14c3ea69419615f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f059dffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f059dffb640 1 -- 172.21.15.110:0/1678101989 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598097e40 con 0x558d63fa7840 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f059dffb640 1 -- 172.21.15.110:0/1678101989 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f05980094f0 con 0x558d63fa7840 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f05b77b3740 1 -- 172.21.15.110:0/1678101989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05980d14e0 con 0x558d63fa7840 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f059dffb640 1 -- 172.21.15.110:0/1678101989 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f059809b500 con 0x558d63fa7840 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f05b77b3740 10 client.26012 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f05b77b3740 1 -- 172.21.15.110:0/1678101989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558d63faf960 con 0x558d63fa7840 2024-04-25T00:17:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f059dffb640 1 -- 172.21.15.110:0/1678101989 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05980d14e0 con 0x558d63fa7840 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.220+0000 7f059dffb640 1 -- 172.21.15.110:0/1678101989 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f059809be00 con 0x558d63fa7840 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f05b77b3740 10 client.26012 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f05b77b3740 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f05b77b3740 20 client.26012 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f05b77b3740 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558d63fc5690 0x558d63fc7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f05b77b3740 4 client.26012 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f05b77b3740 1 -- 172.21.15.110:0/1678101989 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d63f181f0 con 0x558d63fc5690 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f05ae59c640 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558d63fc5690 0x558d63fc7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f059dffb640 10 client.26012 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.221+0000 7f05ae59c640 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558d63fc5690 0x558d63fc7a80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bafe66623f7a4a server_cookie=21c2d58975977a87 in_seq=0 out_seq=0 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.222+0000 7f059dffb640 1 -- 172.21.15.110:0/1678101989 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d63f181f0 con 0x558d63fc5690 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.222+0000 7f059dffb640 10 client.26012 handle_command_reply: tid=0 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.222+0000 7f05b77b3740 1 client.26012 shutdown 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.222+0000 7f05b77b3740 1 -- 172.21.15.110:0/1678101989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa7840 msgr2=0x558d63fba050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.222+0000 7f05b77b3740 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa7840 0x558d63fba050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f059809b8a0 tx=0x7f0598003d00 comp rx=0 tx=0).stop 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.222+0000 7f05b77b3740 1 -- 172.21.15.110:0/1678101989 shutdown_connections 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f05b77b3740 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558d63fc5690 0x558d63fc7a80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f05b77b3740 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fac9b0 0x558d63fc1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f05b77b3740 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa7840 0x558d63fba050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f05b77b3740 1 --2- 172.21.15.110:0/1678101989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa6e50 0x558d63fb9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f05b77b3740 1 -- 172.21.15.110:0/1678101989 >> 172.21.15.110:0/1678101989 conn(0x558d63fa24c0 msgr2=0x558d63fa15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f059dffb640 0 client.26012 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f05b77b3740 1 -- 172.21.15.110:0/1678101989 shutdown_connections 2024-04-25T00:17:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f05b77b3740 1 -- 172.21.15.110:0/1678101989 wait complete. 2024-04-25T00:17:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.223+0000 7f05b77b3740 20 client.26012 trim_cache size 0 max 16384 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.244+0000 7f05b77b3740 1 Processor -- start 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05b77b3740 1 -- start start 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa6e50 0x558d63fa8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa7840 0x558d63fc1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63fa0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05b77b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a91c0 con 0x558d63fa6e50 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05b77b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a6a90 con 0x558d63fac9b0 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05b77b3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a48e0 con 0x558d63fa7840 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05ae59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa6e50 0x558d63fa8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05ae59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa6e50 0x558d63fa8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53306/0 (socket says 172.21.15.110:53306) 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05ae59c640 1 -- 172.21.15.110:0/3212628228 learned_addr learned my addr 172.21.15.110:0/3212628228 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05aed9d640 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63fa0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05a7fff640 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa7840 0x558d63fc1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05ae59c640 1 -- 172.21.15.110:0/3212628228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa7840 msgr2=0x558d63fc1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05ae59c640 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa7840 0x558d63fc1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05ae59c640 1 -- 172.21.15.110:0/3212628228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 msgr2=0x558d63fa0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05ae59c640 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63fa0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05ae59c640 1 -- 172.21.15.110:0/3212628228 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d63fa2060 con 0x558d63fa6e50 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.245+0000 7f05aed9d640 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63fa0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.246+0000 7f05ae59c640 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa6e50 0x558d63fa8160 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f05a00098a0 tx=0x7f05a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c891c3b692e1c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.246+0000 7f059ffff640 1 -- 172.21.15.110:0/3212628228 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0004220 con 0x558d63fa6e50 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.246+0000 7f059ffff640 1 -- 172.21.15.110:0/3212628228 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05a0025070 con 0x558d63fa6e50 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.246+0000 7f059ffff640 1 -- 172.21.15.110:0/3212628228 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0004bd0 con 0x558d63fa6e50 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 -- 172.21.15.110:0/3212628228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa6e50 msgr2=0x558d63fa8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa6e50 0x558d63fa8160 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f05a00098a0 tx=0x7f05a0005f90 comp rx=0 tx=0).stop 2024-04-25T00:17:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 -- 172.21.15.110:0/3212628228 shutdown_connections 2024-04-25T00:17:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fac9b0 0x558d63fa0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fa7840 0x558d63fc1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 --2- 172.21.15.110:0/3212628228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa6e50 0x558d63fa8160 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 -- 172.21.15.110:0/3212628228 >> 172.21.15.110:0/3212628228 conn(0x558d63fa24c0 msgr2=0x558d63fa8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 -- 172.21.15.110:0/3212628228 shutdown_connections 2024-04-25T00:17:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 -- 172.21.15.110:0/3212628228 wait complete. 2024-04-25T00:17:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 -- start start 2024-04-25T00:17:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.247+0000 7f05b77b3740 1 Processor -- start 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa6e50 0x558d63fbe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fbe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05b77b3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fac9b0 0x558d63fbf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05b77b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a4f50 con 0x558d63fa7840 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05b77b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a63c0 con 0x558d63fa6e50 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05b77b3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01a5620 con 0x558d63fac9b0 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05b77b3740 10 client.0 _ll_register_callbacks cb 0x7f05980e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fbe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05ae59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa6e50 0x558d63fbe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fbe840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53316/0 (socket says 172.21.15.110:53316) 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05a7fff640 1 -- 172.21.15.110:0/2193786432 learned_addr learned my addr 172.21.15.110:0/2193786432 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05aed9d640 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fac9b0 0x558d63fbf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05a7fff640 1 -- 172.21.15.110:0/2193786432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fac9b0 msgr2=0x558d63fbf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05a7fff640 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fac9b0 0x558d63fbf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05a7fff640 1 -- 172.21.15.110:0/2193786432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa6e50 msgr2=0x558d63fbe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05a7fff640 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa6e50 0x558d63fbe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.248+0000 7f05a7fff640 1 -- 172.21.15.110:0/2193786432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d63fbfdd0 con 0x558d63fa7840 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.249+0000 7f05a7fff640 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fbe840 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f059809bb30 tx=0x7f0598099ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5408e4d457ac3fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.249+0000 7f059dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.249+0000 7f059dffb640 1 -- 172.21.15.110:0/2193786432 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059809b6e0 con 0x558d63fa7840 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.249+0000 7f059dffb640 1 -- 172.21.15.110:0/2193786432 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f05980a4380 con 0x558d63fa7840 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.249+0000 7f05b77b3740 1 -- 172.21.15.110:0/2193786432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a0025070 con 0x558d63fa7840 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.249+0000 7f059dffb640 1 -- 172.21.15.110:0/2193786432 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f059809b970 con 0x558d63fa7840 2024-04-25T00:17:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.249+0000 7f05b77b3740 10 client.16431 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.249+0000 7f05b77b3740 1 -- 172.21.15.110:0/2193786432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558d63fbfdd0 con 0x558d63fa7840 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.250+0000 7f059dffb640 1 -- 172.21.15.110:0/2193786432 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05a0025070 con 0x558d63fa7840 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.250+0000 7f059dffb640 1 -- 172.21.15.110:0/2193786432 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f05980ba2e0 con 0x558d63fa7840 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.250+0000 7f05b77b3740 10 client.16431 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.250+0000 7f05b77b3740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.250+0000 7f05b77b3740 20 client.16431 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.250+0000 7f05b77b3740 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558d6402bc30 0x558d6402e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.250+0000 7f05b77b3740 4 client.16431 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.250+0000 7f05b77b3740 1 -- 172.21.15.110:0/2193786432 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d63f12540 con 0x558d6402bc30 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.251+0000 7f05ae59c640 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558d6402bc30 0x558d6402e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.251+0000 7f059dffb640 10 client.16431 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.251+0000 7f05ae59c640 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558d6402bc30 0x558d6402e020 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa208c61d558072 server_cookie=44c84c43e0ede329 in_seq=0 out_seq=0 2024-04-25T00:17:05.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:04 smithi188 ceph-mon[33285]: pgmap v802: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 897 op/s 2024-04-25T00:17:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.269+0000 7f059dffb640 1 -- 172.21.15.110:0/2193786432 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d63f12540 con 0x558d6402bc30 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.269+0000 7f059dffb640 10 client.16431 handle_command_reply: tid=0 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.269+0000 7f05b77b3740 1 client.16431 shutdown 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.269+0000 7f05b77b3740 1 -- 172.21.15.110:0/2193786432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 msgr2=0x558d63fbe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.269+0000 7f05b77b3740 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fbe840 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f059809bb30 tx=0x7f0598099ea0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 1 -- 172.21.15.110:0/2193786432 shutdown_connections 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x558d6402bc30 0x558d6402e020 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558d63fac9b0 0x558d63fbf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d63fa7840 0x558d63fbe840 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 1 --2- 172.21.15.110:0/2193786432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558d63fa6e50 0x558d63fbe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 1 -- 172.21.15.110:0/2193786432 >> 172.21.15.110:0/2193786432 conn(0x558d63fa24c0 msgr2=0x558d63fa8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f059dffb640 0 client.16431 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 1 -- 172.21.15.110:0/2193786432 shutdown_connections 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 1 -- 172.21.15.110:0/2193786432 wait complete. 2024-04-25T00:17:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.270+0000 7f05b77b3740 20 client.16431 trim_cache size 0 max 16384 2024-04-25T00:17:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:17:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:17:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:17:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:17:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 -- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05900a7140 msgr2=0x7f05900a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05900a7140 0x7f05900a9600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x558d63fa80a0 tx=0x7f05a8030940 comp rx=0 tx=0).stop 2024-04-25T00:17:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 -- 172.21.15.110:0/982236363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b0102990 msgr2=0x7f05b0197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b0102990 0x7f05b0197e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f059800e9c0 tx=0x7f059800ee80 comp rx=0 tx=0).stop 2024-04-25T00:17:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 -- 172.21.15.110:0/982236363 shutdown_connections 2024-04-25T00:17:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f05900a7140 0x7f05900a9600 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05b0103350 0x7f05b019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b0102990 0x7f05b0197e50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 --2- 172.21.15.110:0/982236363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05b01017a0 0x7f05b0197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 -- 172.21.15.110:0/982236363 >> 172.21.15.110:0/982236363 conn(0x7f05b00fcd10 msgr2=0x7f05b0105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 -- 172.21.15.110:0/982236363 shutdown_connections 2024-04-25T00:17:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:05.271+0000 7f05b4aa5640 1 -- 172.21.15.110:0/982236363 wait complete. 2024-04-25T00:17:05.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:04 smithi110 ceph-mon[29187]: pgmap v802: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 897 op/s 2024-04-25T00:17:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:04 smithi173 ceph-mon[32402]: pgmap v802: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 897 op/s 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 840.0 s, 810.2 tps, lat 122.404 ms stddev 20.849 2024-04-25T00:17:07.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:06 smithi188 ceph-mon[33285]: pgmap v803: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 784 op/s 2024-04-25T00:17:07.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:06 smithi110 ceph-mon[29187]: pgmap v803: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 784 op/s 2024-04-25T00:17:07.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:06 smithi173 ceph-mon[32402]: pgmap v803: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 784 op/s 2024-04-25T00:17:09.153 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:08 smithi173 ceph-mon[32402]: pgmap v804: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 956 op/s 2024-04-25T00:17:09.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:08 smithi188 ceph-mon[33285]: pgmap v804: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 956 op/s 2024-04-25T00:17:09.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:08 smithi110 ceph-mon[29187]: pgmap v804: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 MiB/s wr, 956 op/s 2024-04-25T00:17:11.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:10 smithi188 ceph-mon[33285]: pgmap v805: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 912 op/s 2024-04-25T00:17:11.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:10 smithi110 ceph-mon[29187]: pgmap v805: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 912 op/s 2024-04-25T00:17:11.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:10 smithi173 ceph-mon[32402]: pgmap v805: 97 pgs: 97 active+clean; 1.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 912 op/s 2024-04-25T00:17:11.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 845.0 s, 834.0 tps, lat 121.074 ms stddev 18.350 2024-04-25T00:17:13.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:12 smithi188 ceph-mon[33285]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 MiB/s wr, 986 op/s 2024-04-25T00:17:13.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:12 smithi110 ceph-mon[29187]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 MiB/s wr, 986 op/s 2024-04-25T00:17:13.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:12 smithi173 ceph-mon[32402]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 MiB/s wr, 986 op/s 2024-04-25T00:17:15.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:14 smithi188 ceph-mon[33285]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.9 MiB/s wr, 923 op/s 2024-04-25T00:17:15.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:14 smithi110 ceph-mon[29187]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.9 MiB/s wr, 923 op/s 2024-04-25T00:17:15.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:14 smithi173 ceph-mon[32402]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.9 MiB/s wr, 923 op/s 2024-04-25T00:17:16.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 850.0 s, 818.0 tps, lat 122.153 ms stddev 26.292 2024-04-25T00:17:17.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:16 smithi188 ceph-mon[33285]: pgmap v808: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 817 op/s 2024-04-25T00:17:17.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:16 smithi110 ceph-mon[29187]: pgmap v808: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 817 op/s 2024-04-25T00:17:17.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:16 smithi173 ceph-mon[32402]: pgmap v808: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 817 op/s 2024-04-25T00:17:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.410+0000 7f30baa43640 1 -- 172.21.15.110:0/933998241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b410b0b0 msgr2=0x7f30b410d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.410+0000 7f30baa43640 1 --2- 172.21.15.110:0/933998241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b410b0b0 0x7f30b410d520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009be0 tx=0x7f30ac02a6c0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.411+0000 7f30baa43640 1 -- 172.21.15.110:0/933998241 shutdown_connections 2024-04-25T00:17:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.411+0000 7f30baa43640 1 --2- 172.21.15.110:0/933998241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b410da60 0x7f30b410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.411+0000 7f30baa43640 1 --2- 172.21.15.110:0/933998241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b410b0b0 0x7f30b410d520 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.411+0000 7f30baa43640 1 --2- 172.21.15.110:0/933998241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30b4108740 0x7f30b410ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.411+0000 7f30baa43640 1 -- 172.21.15.110:0/933998241 >> 172.21.15.110:0/933998241 conn(0x7f30b406ef20 msgr2=0x7f30b4071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.411+0000 7f30baa43640 1 -- 172.21.15.110:0/933998241 shutdown_connections 2024-04-25T00:17:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.411+0000 7f30baa43640 1 -- 172.21.15.110:0/933998241 wait complete. 2024-04-25T00:17:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.412+0000 7f30baa43640 1 Processor -- start 2024-04-25T00:17:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.412+0000 7f30baa43640 1 -- start start 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.412+0000 7f30baa43640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b4108740 0x7f30b41a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.412+0000 7f30baa43640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30b410b0b0 0x7f30b41a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.412+0000 7f30baa43640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b410da60 0x7f30b41adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.412+0000 7f30baa43640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f30b411bec0 con 0x7f30b410b0b0 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.412+0000 7f30baa43640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f30b411bd40 con 0x7f30b4108740 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30baa43640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f30b411c040 con 0x7f30b410da60 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.412+0000 7f30b9a41640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b4108740 0x7f30b41a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9a41640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b4108740 0x7f30b41a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:54760/0 (socket says 172.21.15.110:54760) 2024-04-25T00:17:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9a41640 1 -- 172.21.15.110:0/314937334 learned_addr learned my addr 172.21.15.110:0/314937334 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9240640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30b410b0b0 0x7f30b41a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30ba242640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b410da60 0x7f30b41adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9a41640 1 -- 172.21.15.110:0/314937334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b410da60 msgr2=0x7f30b41adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9a41640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b410da60 0x7f30b41adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9a41640 1 -- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30b410b0b0 msgr2=0x7f30b41a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9a41640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30b410b0b0 0x7f30b41a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9a41640 1 -- 172.21.15.110:0/314937334 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b41ae210 con 0x7f30b4108740 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30b9a41640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b4108740 0x7f30b41a65f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f30a400d950 tx=0x7f30a400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.413+0000 7f30aaffd640 1 -- 172.21.15.110:0/314937334 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4017030 con 0x7f30b4108740 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.414+0000 7f30baa43640 1 -- 172.21.15.110:0/314937334 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b41ae500 con 0x7f30b4108740 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.414+0000 7f30aaffd640 1 -- 172.21.15.110:0/314937334 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30a4004020 con 0x7f30b4108740 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.414+0000 7f30aaffd640 1 -- 172.21.15.110:0/314937334 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a40047a0 con 0x7f30b4108740 2024-04-25T00:17:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.414+0000 7f30baa43640 1 -- 172.21.15.110:0/314937334 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b41aea70 con 0x7f30b4108740 2024-04-25T00:17:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.417+0000 7f30aaffd640 1 -- 172.21.15.110:0/314937334 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f30a40041d0 con 0x7f30b4108740 2024-04-25T00:17:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.418+0000 7f30be9bc740 1 Processor -- start 2024-04-25T00:17:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30be9bc740 1 -- start start 2024-04-25T00:17:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 0x55eba976e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba9773520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 0x55eba9776020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30be9bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eba96d1cb0 con 0x55eba9773b90 2024-04-25T00:17:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30be9bc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eba967a5b0 con 0x55eba976ea20 2024-04-25T00:17:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30be9bc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eba96cdc70 con 0x55eba976e030 2024-04-25T00:17:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30b9a41640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba9773520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30ba242640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 0x55eba976e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30b9a41640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba9773520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:54766/0 (socket says 172.21.15.110:54766) 2024-04-25T00:17:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.419+0000 7f30b9a41640 1 -- 172.21.15.110:0/549724202 learned_addr learned my addr 172.21.15.110:0/549724202 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.420+0000 7f30aaffd640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f30a00a2c00 0x7f30a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.420+0000 7f30b9240640 1 --2- 172.21.15.110:0/549724202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 0x55eba9776020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.420+0000 7f30aaffd640 1 -- 172.21.15.110:0/314937334 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f30a40ceb50 con 0x7f30b4108740 2024-04-25T00:17:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.420+0000 7f30b9a41640 1 -- 172.21.15.110:0/549724202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 msgr2=0x55eba976e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.420+0000 7f30b9a41640 1 --2- 172.21.15.110:0/549724202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 0x55eba976e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f30b9a41640 1 -- 172.21.15.110:0/549724202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 msgr2=0x55eba9776020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f30b9a41640 1 --2- 172.21.15.110:0/549724202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 0x55eba9776020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f30b9a41640 1 -- 172.21.15.110:0/549724202 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba93e6210 con 0x55eba976ea20 2024-04-25T00:17:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f30ba242640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f30a00a2c00 0x7f30a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f30b9a41640 1 --2- 172.21.15.110:0/549724202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba9773520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f30a40977e0 tx=0x7f30a4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7baa10d245c65b71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f309ffff640 1 -- 172.21.15.110:0/549724202 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a409b380 con 0x55eba976ea20 2024-04-25T00:17:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f309ffff640 1 -- 172.21.15.110:0/549724202 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30a409f730 con 0x55eba976ea20 2024-04-25T00:17:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f309ffff640 1 -- 172.21.15.110:0/549724202 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4009c90 con 0x55eba976ea20 2024-04-25T00:17:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.421+0000 7f30ba242640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f30a00a2c00 0x7f30a00a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55eba976f280 tx=0x7f30b0014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.422+0000 7f30be9bc740 1 -- 172.21.15.110:0/549724202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 msgr2=0x55eba9773520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.422+0000 7f30be9bc740 1 --2- 172.21.15.110:0/549724202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba9773520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f30a40977e0 tx=0x7f30a4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.423+0000 7f30be9bc740 1 -- 172.21.15.110:0/549724202 shutdown_connections 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.423+0000 7f30be9bc740 1 --2- 172.21.15.110:0/549724202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 0x55eba9776020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.423+0000 7f30be9bc740 1 --2- 172.21.15.110:0/549724202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba9773520 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.423+0000 7f30be9bc740 1 --2- 172.21.15.110:0/549724202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 0x55eba976e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.423+0000 7f30be9bc740 1 -- 172.21.15.110:0/549724202 >> 172.21.15.110:0/549724202 conn(0x55eba97696a0 msgr2=0x55eba9769aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.423+0000 7f30be9bc740 1 -- 172.21.15.110:0/549724202 shutdown_connections 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.423+0000 7f30be9bc740 1 -- 172.21.15.110:0/549724202 wait complete. 2024-04-25T00:17:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.423+0000 7f30be9bc740 1 -- start start 2024-04-25T00:17:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.424+0000 7f30be9bc740 1 Processor -- start 2024-04-25T00:17:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.424+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9789280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.424+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30b9240640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30b9240640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97897c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:45832/0 (socket says 172.21.15.110:45832) 2024-04-25T00:17:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30b9240640 1 -- 172.21.15.110:0/4250746198 learned_addr learned my addr 172.21.15.110:0/4250746198 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30b9a41640 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9789280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30be9bc740 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9790800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eba9787af0 con 0x55eba976e030 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30ba242640 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9790800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eba968db50 con 0x55eba9773b90 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eba9787c70 con 0x55eba976ea20 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.425+0000 7f30be9bc740 10 client.0 _ll_register_callbacks cb 0x55eba96c8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.426+0000 7f30ba242640 1 -- 172.21.15.110:0/4250746198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 msgr2=0x55eba97897c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.426+0000 7f30ba242640 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97897c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.426+0000 7f30ba242640 1 -- 172.21.15.110:0/4250746198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 msgr2=0x55eba9789280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.426+0000 7f30ba242640 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9789280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.426+0000 7f30ba242640 1 -- 172.21.15.110:0/4250746198 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba94b2620 con 0x55eba9773b90 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.426+0000 7f30ba242640 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9790800 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f30b0005fd0 tx=0x7f30b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cca4f3fd90a0e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.426+0000 7f309d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.427+0000 7f309d7fa640 1 -- 172.21.15.110:0/4250746198 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0027420 con 0x55eba9773b90 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.427+0000 7f309d7fa640 1 -- 172.21.15.110:0/4250746198 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f30b0010420 con 0x55eba9773b90 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.427+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a409f730 con 0x55eba9773b90 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f309d7fa640 1 -- 172.21.15.110:0/4250746198 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30b0010620 con 0x55eba9773b90 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f30be9bc740 10 client.26030 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eba9793bc0 con 0x55eba9773b90 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f309d7fa640 1 -- 172.21.15.110:0/4250746198 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30b0010c70 con 0x55eba9773b90 2024-04-25T00:17:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f309d7fa640 1 -- 172.21.15.110:0/4250746198 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f30b0027b60 con 0x55eba9773b90 2024-04-25T00:17:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f30be9bc740 10 client.26030 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f30be9bc740 10 client.26030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f30be9bc740 20 client.26030 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f30be9bc740 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eba9795250 0x55eba97976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f30be9bc740 4 client.26030 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.428+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f30a409f730 con 0x55eba9795250 2024-04-25T00:17:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.429+0000 7f30b9a41640 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eba9795250 0x55eba97976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.429+0000 7f309d7fa640 10 client.26030 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.429+0000 7f30b9a41640 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eba9795250 0x55eba97976b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb445b669c8bf4a6 server_cookie=2aaec55bc5838e07 in_seq=0 out_seq=0 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.430+0000 7f309d7fa640 1 -- 172.21.15.110:0/4250746198 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f30a409f730 con 0x55eba9795250 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.430+0000 7f309d7fa640 10 client.26030 handle_command_reply: tid=0 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.430+0000 7f30be9bc740 1 client.26030 shutdown 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.430+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 msgr2=0x55eba9790800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.430+0000 7f30be9bc740 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9790800 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f30b0005fd0 tx=0x7f30b00046c0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 shutdown_connections 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eba9795250 0x55eba97976b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9790800 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97897c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 1 --2- 172.21.15.110:0/4250746198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9789280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 >> 172.21.15.110:0/4250746198 conn(0x55eba97696a0 msgr2=0x55eba97688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f309d7fa640 0 client.26030 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 shutdown_connections 2024-04-25T00:17:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 1 -- 172.21.15.110:0/4250746198 wait complete. 2024-04-25T00:17:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.431+0000 7f30be9bc740 20 client.26030 trim_cache size 0 max 16384 2024-04-25T00:17:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30be9bc740 1 Processor -- start 2024-04-25T00:17:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30be9bc740 1 -- start start 2024-04-25T00:17:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9797260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9789280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30be9bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f30b41b1930 con 0x55eba976e030 2024-04-25T00:17:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30be9bc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f30b41af200 con 0x55eba9773b90 2024-04-25T00:17:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30be9bc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4116990 con 0x55eba976ea20 2024-04-25T00:17:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30b9240640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30b9a41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9797260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30b9240640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97907c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:45836/0 (socket says 172.21.15.110:45836) 2024-04-25T00:17:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30b9240640 1 -- 172.21.15.110:0/2667790042 learned_addr learned my addr 172.21.15.110:0/2667790042 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.452+0000 7f30ba242640 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9789280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f30b9240640 1 -- 172.21.15.110:0/2667790042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 msgr2=0x55eba9789280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f30b9240640 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9789280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f30b9240640 1 -- 172.21.15.110:0/2667790042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 msgr2=0x55eba9797260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f30b9240640 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9797260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f30b9240640 1 -- 172.21.15.110:0/2667790042 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba97977a0 con 0x55eba976ea20 2024-04-25T00:17:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f30b9a41640 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9797260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f30b9240640 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97907c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f30ac02acb0 tx=0x7f30ac030670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7856690680ba730f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f309f7fe640 1 -- 172.21.15.110:0/2667790042 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac036040 con 0x55eba976ea20 2024-04-25T00:17:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f309f7fe640 1 -- 172.21.15.110:0/2667790042 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30ac037040 con 0x55eba976ea20 2024-04-25T00:17:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.453+0000 7f309f7fe640 1 -- 172.21.15.110:0/2667790042 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac004590 con 0x55eba976ea20 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.454+0000 7f30be9bc740 1 -- 172.21.15.110:0/2667790042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 msgr2=0x55eba97907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.454+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97907c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f30ac02acb0 tx=0x7f30ac030670 comp rx=0 tx=0).stop 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 -- 172.21.15.110:0/2667790042 shutdown_connections 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba9773b90 0x55eba9789280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976ea20 0x55eba97907c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2667790042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba976e030 0x55eba9797260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 -- 172.21.15.110:0/2667790042 >> 172.21.15.110:0/2667790042 conn(0x55eba97696a0 msgr2=0x55eba9769a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 -- 172.21.15.110:0/2667790042 shutdown_connections 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 -- 172.21.15.110:0/2667790042 wait complete. 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 -- start start 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.455+0000 7f30be9bc740 1 Processor -- start 2024-04-25T00:17:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 0x55eba978d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba978db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30be9bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 0x55eba97f9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30be9bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4117000 con 0x55eba9773b90 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30be9bc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4118470 con 0x55eba976ea20 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30be9bc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f30b41176d0 con 0x55eba976e030 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba978db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9a41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 0x55eba978d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30be9bc740 10 client.0 _ll_register_callbacks cb 0x7f30b0004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba978db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:54804/0 (socket says 172.21.15.110:54804) 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 -- 172.21.15.110:0/2457636281 learned_addr learned my addr 172.21.15.110:0/2457636281 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30ba242640 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 0x55eba97f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 -- 172.21.15.110:0/2457636281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 msgr2=0x55eba978d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 0x55eba978d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 -- 172.21.15.110:0/2457636281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 msgr2=0x55eba97f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 0x55eba97f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 -- 172.21.15.110:0/2457636281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba97f9d60 con 0x55eba976ea20 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f30b9240640 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba978db90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f30ac034040 tx=0x7f30ac005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58311b8e9f78171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.456+0000 7f309d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f309d7fa640 1 -- 172.21.15.110:0/2457636281 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac036040 con 0x55eba976ea20 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f309d7fa640 1 -- 172.21.15.110:0/2457636281 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f30ac030df0 con 0x55eba976ea20 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 1 -- 172.21.15.110:0/2457636281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30ac037040 con 0x55eba976ea20 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f309d7fa640 1 -- 172.21.15.110:0/2457636281 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30ac003b80 con 0x55eba976ea20 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 10 client.26036 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 1 -- 172.21.15.110:0/2457636281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eba97fafa0 con 0x55eba976ea20 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f309d7fa640 1 -- 172.21.15.110:0/2457636281 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30ac037040 con 0x55eba976ea20 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f309d7fa640 1 -- 172.21.15.110:0/2457636281 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f30ac058030 con 0x55eba976ea20 2024-04-25T00:17:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 10 client.26036 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 20 client.26036 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eba97fb920 0x55eba97fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 4 client.26036 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:17:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.457+0000 7f30be9bc740 1 -- 172.21.15.110:0/2457636281 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eba96df1f0 con 0x55eba97fb920 2024-04-25T00:17:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.458+0000 7f30b9a41640 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eba97fb920 0x55eba97fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.458+0000 7f309d7fa640 10 client.26036 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.458+0000 7f30b9a41640 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eba97fb920 0x55eba97fdd40 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b559a6da4c4ed1df server_cookie=401f44ed09507be1 in_seq=0 out_seq=0 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f309d7fa640 1 -- 172.21.15.110:0/2457636281 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eba96df1f0 con 0x55eba97fb920 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f309d7fa640 10 client.26036 handle_command_reply: tid=0 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 client.26036 shutdown 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 -- 172.21.15.110:0/2457636281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 msgr2=0x55eba978db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba978db90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f30ac034040 tx=0x7f30ac005fd0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 -- 172.21.15.110:0/2457636281 shutdown_connections 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55eba97fb920 0x55eba97fdd40 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f309d7fa640 0 client.26036 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eba9773b90 0x55eba97f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eba976ea20 0x55eba978db90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 --2- 172.21.15.110:0/2457636281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba976e030 0x55eba978d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 -- 172.21.15.110:0/2457636281 >> 172.21.15.110:0/2457636281 conn(0x55eba97696a0 msgr2=0x55eba976fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 -- 172.21.15.110:0/2457636281 shutdown_connections 2024-04-25T00:17:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 1 -- 172.21.15.110:0/2457636281 wait complete. 2024-04-25T00:17:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.478+0000 7f30be9bc740 20 client.26036 trim_cache size 0 max 16384 2024-04-25T00:17:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:17:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:17:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:17:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 -- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f30a00a2c00 msgr2=0x7f30a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f30a00a2c00 0x7f30a00a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55eba976f280 tx=0x7f30b0014040 comp rx=0 tx=0).stop 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 -- 172.21.15.110:0/314937334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b4108740 msgr2=0x7f30b41a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b4108740 0x7f30b41a65f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f30a400d950 tx=0x7f30a400de10 comp rx=0 tx=0).stop 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 -- 172.21.15.110:0/314937334 shutdown_connections 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f30a00a2c00 0x7f30a00a50c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b410da60 0x7f30b41adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30b410b0b0 0x7f30b41a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 --2- 172.21.15.110:0/314937334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30b4108740 0x7f30b41a65f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 -- 172.21.15.110:0/314937334 >> 172.21.15.110:0/314937334 conn(0x7f30b406ef20 msgr2=0x7f30b4108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 -- 172.21.15.110:0/314937334 shutdown_connections 2024-04-25T00:17:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:18.479+0000 7f30baa43640 1 -- 172.21.15.110:0/314937334 wait complete. 2024-04-25T00:17:19.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:18 smithi188 ceph-mon[33285]: pgmap v809: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 974 op/s 2024-04-25T00:17:19.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:18 smithi110 ceph-mon[29187]: pgmap v809: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 974 op/s 2024-04-25T00:17:19.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:18 smithi173 ceph-mon[32402]: pgmap v809: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 974 op/s 2024-04-25T00:17:21.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:20 smithi188 ceph-mon[33285]: pgmap v810: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 883 op/s 2024-04-25T00:17:21.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:20 smithi110 ceph-mon[29187]: pgmap v810: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 883 op/s 2024-04-25T00:17:21.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:20 smithi173 ceph-mon[32402]: pgmap v810: 97 pgs: 97 active+clean; 1.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 883 op/s 2024-04-25T00:17:21.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 855.0 s, 782.4 tps, lat 127.179 ms stddev 23.358 2024-04-25T00:17:23.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:22 smithi188 ceph-mon[33285]: pgmap v811: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 963 op/s 2024-04-25T00:17:23.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:22 smithi110 ceph-mon[29187]: pgmap v811: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 963 op/s 2024-04-25T00:17:23.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:22 smithi173 ceph-mon[32402]: pgmap v811: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 963 op/s 2024-04-25T00:17:25.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:24 smithi188 ceph-mon[33285]: pgmap v812: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 839 op/s 2024-04-25T00:17:25.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:24 smithi110 ceph-mon[29187]: pgmap v812: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 839 op/s 2024-04-25T00:17:25.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:24 smithi173 ceph-mon[32402]: pgmap v812: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 839 op/s 2024-04-25T00:17:26.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 860.0 s, 824.6 tps, lat 121.450 ms stddev 15.832 2024-04-25T00:17:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:26 smithi188 ceph-mon[33285]: pgmap v813: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-04-25T00:17:27.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:26 smithi110 ceph-mon[29187]: pgmap v813: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-04-25T00:17:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:26 smithi173 ceph-mon[32402]: pgmap v813: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-04-25T00:17:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:28 smithi188 ceph-mon[33285]: pgmap v814: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-04-25T00:17:29.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:28 smithi110 ceph-mon[29187]: pgmap v814: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-04-25T00:17:29.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:28 smithi173 ceph-mon[32402]: pgmap v814: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-04-25T00:17:31.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:30 smithi188 ceph-mon[33285]: pgmap v815: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 894 op/s 2024-04-25T00:17:31.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:30 smithi110 ceph-mon[29187]: pgmap v815: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 894 op/s 2024-04-25T00:17:31.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:30 smithi173 ceph-mon[32402]: pgmap v815: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 894 op/s 2024-04-25T00:17:31.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 865.0 s, 800.6 tps, lat 124.600 ms stddev 20.557 2024-04-25T00:17:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/466333970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f281006c0 msgr2=0x7f6f28100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/466333970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f281006c0 0x7f6f28100ae0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009900 tx=0x7f6f1802a410 comp rx=0 tx=0).stop 2024-04-25T00:17:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/466333970 shutdown_connections 2024-04-25T00:17:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/466333970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f28102270 0x7f6f28108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/466333970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f281018b0 0x7f6f28101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/466333970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f281006c0 0x7f6f28100ae0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/466333970 >> 172.21.15.110:0/466333970 conn(0x7f6f280fbc50 msgr2=0x7f6f280fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/466333970 shutdown_connections 2024-04-25T00:17:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.613+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/466333970 wait complete. 2024-04-25T00:17:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2c9c1640 1 Processor -- start 2024-04-25T00:17:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2c9c1640 1 -- start start 2024-04-25T00:17:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2c9c1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f281006c0 0x7f6f2819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2c9c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f281018b0 0x7f6f2819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2c9c1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f28102270 0x7f6f281a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f281006c0 0x7f6f2819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2c9c1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2810b0f0 con 0x7f6f281006c0 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2c9c1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2810af70 con 0x7f6f28102270 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2c9c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2810b270 con 0x7f6f281018b0 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f281006c0 0x7f6f2819e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33842/0 (socket says 172.21.15.110:33842) 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f2659c640 1 -- 172.21.15.110:0/158622696 learned_addr learned my addr 172.21.15.110:0/158622696 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f28102270 0x7f6f281a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.614+0000 7f6f1ffff640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f281018b0 0x7f6f2819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f26d9d640 1 -- 172.21.15.110:0/158622696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f281018b0 msgr2=0x7f6f2819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f281018b0 0x7f6f2819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f26d9d640 1 -- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f281006c0 msgr2=0x7f6f2819e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f281006c0 0x7f6f2819e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f26d9d640 1 -- 172.21.15.110:0/158622696 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f281a5b50 con 0x7f6f28102270 2024-04-25T00:17:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f28102270 0x7f6f281a54e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009fd0 tx=0x7f6f2000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f1f7fe640 1 -- 172.21.15.110:0/158622696 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000ce20 con 0x7f6f28102270 2024-04-25T00:17:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f1f7fe640 1 -- 172.21.15.110:0/158622696 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f20005020 con 0x7f6f28102270 2024-04-25T00:17:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f1f7fe640 1 -- 172.21.15.110:0/158622696 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f20010690 con 0x7f6f28102270 2024-04-25T00:17:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/158622696 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f281a5e40 con 0x7f6f28102270 2024-04-25T00:17:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.615+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/158622696 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f281a6250 con 0x7f6f28102270 2024-04-25T00:17:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.618+0000 7f6f1f7fe640 1 -- 172.21.15.110:0/158622696 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6f20004b20 con 0x7f6f28102270 2024-04-25T00:17:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.619+0000 7f6f1f7fe640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6f140a2c30 0x7f6f140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.619+0000 7f6f2f7ab740 1 Processor -- start 2024-04-25T00:17:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.619+0000 7f6f2659c640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6f140a2c30 0x7f6f140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.619+0000 7f6f2f7ab740 1 -- start start 2024-04-25T00:17:31.621 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f1f7fe640 1 -- 172.21.15.110:0/158622696 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6f200cf650 con 0x7f6f28102270 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a53340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a55e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2f7ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5649719b1cb0 con 0x564971a4e840 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a53340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a53340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33858/0 (socket says 172.21.15.110:33858) 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f1ffff640 1 -- 172.21.15.110:0/733476320 learned_addr learned my addr 172.21.15.110:0/733476320 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48648/0 (socket says 172.21.15.110:48648) 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/733476320 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56497195a5b0 con 0x564971a539b0 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/733476320 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5649719adc70 con 0x564971a4de50 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a55e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2659c640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6f140a2c30 0x7f6f140a50f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6f180096c0 tx=0x7f6f18005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2659c640 1 -- 172.21.15.110:0/733476320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 msgr2=0x564971a55e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2659c640 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a55e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2659c640 1 -- 172.21.15.110:0/733476320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 msgr2=0x564971a53340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.620+0000 7f6f2659c640 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a53340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.621+0000 7f6f2659c640 1 -- 172.21.15.110:0/733476320 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649716c6210 con 0x564971a4de50 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.621+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a55e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.621+0000 7f6f1ffff640 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a53340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.621+0000 7f6f2659c640 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4e270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f18000c00 tx=0x7f6f1803e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3037e65c8b0aa0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.621+0000 7f6f1cff9640 1 -- 172.21.15.110:0/733476320 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18035050 con 0x564971a4de50 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.621+0000 7f6f1cff9640 1 -- 172.21.15.110:0/733476320 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f18051930 con 0x564971a4de50 2024-04-25T00:17:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.621+0000 7f6f1cff9640 1 -- 172.21.15.110:0/733476320 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1803c890 con 0x564971a4de50 2024-04-25T00:17:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/733476320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 msgr2=0x564971a4e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4e270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f18000c00 tx=0x7f6f1803e000 comp rx=0 tx=0).stop 2024-04-25T00:17:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/733476320 shutdown_connections 2024-04-25T00:17:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a55e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a53340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/733476320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4e270 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/733476320 >> 172.21.15.110:0/733476320 conn(0x564971a494c0 msgr2=0x564971a498c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/733476320 shutdown_connections 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/733476320 wait complete. 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 -- start start 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.622+0000 7f6f2f7ab740 1 Processor -- start 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4e840 0x564971a587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a539b0 0x564971a5f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4e840 0x564971a587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:59282/0 (socket says 172.21.15.110:59282) 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4e840 0x564971a587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48664/0 (socket says 172.21.15.110:48664) 2024-04-25T00:17:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f26d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a539b0 0x564971a5f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f26d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a539b0 0x564971a5f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33866/0 (socket says 172.21.15.110:33866) 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2659c640 1 -- 172.21.15.110:0/1255884206 learned_addr learned my addr 172.21.15.110:0/1255884206 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564971a56900 con 0x564971a539b0 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56497196db50 con 0x564971a4de50 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.623+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564971a56a80 con 0x564971a4e840 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f1ffff640 1 -- 172.21.15.110:0/1255884206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 msgr2=0x564971a582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f1ffff640 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f1ffff640 1 -- 172.21.15.110:0/1255884206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a539b0 msgr2=0x564971a5f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f1ffff640 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a539b0 0x564971a5f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f1ffff640 1 -- 172.21.15.110:0/1255884206 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564971792620 con 0x564971a4e840 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f2f7ab740 10 client.0 _ll_register_callbacks cb 0x5649719a8ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f18051930 con 0x564971a4e840 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f1ffff640 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4e840 0x564971a587e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x564971a4ec40 tx=0x7f6f10012b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03f0a20946b470d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f0dffb640 10 client.26041 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:17:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f0dffb640 1 -- 172.21.15.110:0/1255884206 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10004030 con 0x564971a4e840 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f0dffb640 1 -- 172.21.15.110:0/1255884206 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6f100041d0 con 0x564971a4e840 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f0dffb640 1 -- 172.21.15.110:0/1255884206 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f10004410 con 0x564971a4e840 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f2f7ab740 10 client.26041 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.624+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=32}) -- 0x564971a62a90 con 0x564971a4e840 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.625+0000 7f6f0dffb640 1 -- 172.21.15.110:0/1255884206 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6f1000f890 con 0x564971a4e840 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.625+0000 7f6f2f7ab740 10 client.26041 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.625+0000 7f6f2f7ab740 10 client.26041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.625+0000 7f6f2f7ab740 20 client.26041 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.625+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564971a64130 0x564971a665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.625+0000 7f6f2f7ab740 4 client.26041 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.625+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649719bf1f0 con 0x564971a64130 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.625+0000 7f6f2659c640 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564971a64130 0x564971a665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.626+0000 7f6f2659c640 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564971a64130 0x564971a665c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ef30cc851f7c1d server_cookie=341e7994a906db2f in_seq=0 out_seq=0 2024-04-25T00:17:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.626+0000 7f6f0dffb640 10 client.26041 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.626+0000 7f6f0dffb640 1 -- 172.21.15.110:0/1255884206 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649719bf1f0 con 0x564971a64130 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.626+0000 7f6f0dffb640 10 client.26041 handle_command_reply: tid=0 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.626+0000 7f6f2f7ab740 1 client.26041 shutdown 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.626+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4e840 msgr2=0x564971a587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.626+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4e840 0x564971a587e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x564971a4ec40 tx=0x7f6f10012b40 comp rx=0 tx=0).stop 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 shutdown_connections 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564971a64130 0x564971a665c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f0dffb640 0 client.26041 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a539b0 0x564971a5f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4e840 0x564971a587e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/1255884206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 >> 172.21.15.110:0/1255884206 conn(0x564971a494c0 msgr2=0x564971a485c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 shutdown_connections 2024-04-25T00:17:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/1255884206 wait complete. 2024-04-25T00:17:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.627+0000 7f6f2f7ab740 20 client.26041 trim_cache size 0 max 16384 2024-04-25T00:17:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2f7ab740 1 Processor -- start 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2f7ab740 1 -- start start 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a4c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48672/0 (socket says 172.21.15.110:48672) 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33868/0 (socket says 172.21.15.110:33868) 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2659c640 1 -- 172.21.15.110:0/3822958236 learned_addr learned my addr 172.21.15.110:0/3822958236 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3822958236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f281b1890 con 0x564971a4e840 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3822958236 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6f281af160 con 0x564971a539b0 2024-04-25T00:17:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3822958236 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f281acfb0 con 0x564971a4de50 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.651+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a4c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f1ffff640 1 -- 172.21.15.110:0/3822958236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 msgr2=0x564971a4c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f1ffff640 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f1ffff640 1 -- 172.21.15.110:0/3822958236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 msgr2=0x564971a4c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f1ffff640 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a4c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f1ffff640 1 -- 172.21.15.110:0/3822958236 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564971a66ba0 con 0x564971a4e840 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f2659c640 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f1ffff640 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5f7e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6f10001eb0 tx=0x7f6f10014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e2161f44b051f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f0ffff640 1 -- 172.21.15.110:0/3822958236 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000f990 con 0x564971a4e840 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f0ffff640 1 -- 172.21.15.110:0/3822958236 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f18051930 con 0x564971a4e840 2024-04-25T00:17:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.652+0000 7f6f0ffff640 1 -- 172.21.15.110:0/3822958236 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10005350 con 0x564971a4e840 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3822958236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 msgr2=0x564971a5f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5f7e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6f10001eb0 tx=0x7f6f10014000 comp rx=0 tx=0).stop 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3822958236 shutdown_connections 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a539b0 0x564971a4c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5f7e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3822958236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a4de50 0x564971a4c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3822958236 >> 172.21.15.110:0/3822958236 conn(0x564971a494c0 msgr2=0x564971a498a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3822958236 shutdown_connections 2024-04-25T00:17:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.653+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3822958236 wait complete. 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.654+0000 7f6f2f7ab740 1 -- start start 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.654+0000 7f6f2f7ab740 1 Processor -- start 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.654+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a5d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.654+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f2f7ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a539b0 0x564971a5ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f2f7ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f281ad620 con 0x564971a4e840 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f2f7ab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6f281aea90 con 0x564971a4de50 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f2f7ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f281adcf0 con 0x564971a539b0 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f2f7ab740 10 client.0 _ll_register_callbacks cb 0x7f6f10009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33872/0 (socket says 172.21.15.110:33872) 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f1ffff640 1 -- 172.21.15.110:0/3742232492 learned_addr learned my addr 172.21.15.110:0/3742232492 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a539b0 0x564971a5ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f2659c640 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a5d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f26d9d640 1 -- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 msgr2=0x564971a5d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a5d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f26d9d640 1 -- 172.21.15.110:0/3742232492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 msgr2=0x564971a5da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f26d9d640 1 -- 172.21.15.110:0/3742232492 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564971a5f080 con 0x564971a539b0 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.655+0000 7f6f2659c640 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a5d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.656+0000 7f6f26d9d640 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a539b0 0x564971a5ea90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000c760 tx=0x7f6f20098200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7595d291a811136a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.656+0000 7f6f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.656+0000 7f6f0dffb640 1 -- 172.21.15.110:0/3742232492 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f200051d0 con 0x564971a539b0 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.656+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3742232492 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f18051930 con 0x564971a539b0 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.656+0000 7f6f0dffb640 1 -- 172.21.15.110:0/3742232492 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f6f200a46c0 con 0x564971a539b0 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.656+0000 7f6f0dffb640 1 -- 172.21.15.110:0/3742232492 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f20081070 con 0x564971a539b0 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.656+0000 7f6f2f7ab740 10 client.26047 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.656+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3742232492 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564971acad80 con 0x564971a539b0 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f0dffb640 1 -- 172.21.15.110:0/3742232492 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f18051930 con 0x564971a539b0 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f0dffb640 1 -- 172.21.15.110:0/3742232492 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6f20097c30 con 0x564971a539b0 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f2f7ab740 10 client.26047 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f2f7ab740 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f2f7ab740 20 client.26047 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564971a64130 0x564971acd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f2f7ab740 4 client.26047 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3742232492 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649719b9540 con 0x564971a64130 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.657+0000 7f6f2659c640 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564971a64130 0x564971acd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.658+0000 7f6f0dffb640 10 client.26047 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.658+0000 7f6f2659c640 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564971a64130 0x564971acd730 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e91dac7395a9cb server_cookie=d7c58b5ef56f5447 in_seq=0 out_seq=0 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f0dffb640 1 -- 172.21.15.110:0/3742232492 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5649719b9540 con 0x564971a64130 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f0dffb640 10 client.26047 handle_command_reply: tid=0 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 client.26047 shutdown 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3742232492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a539b0 msgr2=0x564971a5ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a539b0 0x564971a5ea90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000c760 tx=0x7f6f20098200 comp rx=0 tx=0).stop 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3742232492 shutdown_connections 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x564971a64130 0x564971acd730 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564971a539b0 0x564971a5ea90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564971a4e840 0x564971a5da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 --2- 172.21.15.110:0/3742232492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564971a4de50 0x564971a5d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3742232492 >> 172.21.15.110:0/3742232492 conn(0x564971a494c0 msgr2=0x564971a4c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f0dffb640 0 client.26047 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3742232492 shutdown_connections 2024-04-25T00:17:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 1 -- 172.21.15.110:0/3742232492 wait complete. 2024-04-25T00:17:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.676+0000 7f6f2f7ab740 20 client.26047 trim_cache size 0 max 16384 2024-04-25T00:17:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:17:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:17:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:17:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:17:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6f140a2c30 msgr2=0x7f6f140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6f140a2c30 0x7f6f140a50f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6f180096c0 tx=0x7f6f18005dd0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/158622696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f28102270 msgr2=0x7f6f281a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f28102270 0x7f6f281a54e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009fd0 tx=0x7f6f2000ef10 comp rx=0 tx=0).stop 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/158622696 shutdown_connections 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f6f140a2c30 0x7f6f140a50f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f28102270 0x7f6f281a54e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f281018b0 0x7f6f2819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 --2- 172.21.15.110:0/158622696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f281006c0 0x7f6f2819e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/158622696 >> 172.21.15.110:0/158622696 conn(0x7f6f280fbc50 msgr2=0x7f6f2810dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/158622696 shutdown_connections 2024-04-25T00:17:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:31.678+0000 7f6f2c9c1640 1 -- 172.21.15.110:0/158622696 wait complete. 2024-04-25T00:17:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:32 smithi188 ceph-mon[33285]: pgmap v816: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:17:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:32 smithi110 ceph-mon[29187]: pgmap v816: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:17:33.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:32 smithi173 ceph-mon[32402]: pgmap v816: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-04-25T00:17:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:34 smithi188 ceph-mon[33285]: pgmap v817: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 873 op/s 2024-04-25T00:17:35.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:34 smithi110 ceph-mon[29187]: pgmap v817: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 873 op/s 2024-04-25T00:17:35.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:34 smithi173 ceph-mon[32402]: pgmap v817: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 12 MiB/s wr, 873 op/s 2024-04-25T00:17:36.590 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 870.0 s, 852.8 tps, lat 117.484 ms stddev 22.829 2024-04-25T00:17:37.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:36 smithi188 ceph-mon[33285]: pgmap v818: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-04-25T00:17:37.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:36 smithi110 ceph-mon[29187]: pgmap v818: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-04-25T00:17:37.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:36 smithi173 ceph-mon[32402]: pgmap v818: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 847 op/s 2024-04-25T00:17:39.153 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:38 smithi173 ceph-mon[32402]: pgmap v819: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:17:39.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:38 smithi188 ceph-mon[33285]: pgmap v819: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:17:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:38 smithi110 ceph-mon[29187]: pgmap v819: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-04-25T00:17:41.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:40 smithi188 ceph-mon[33285]: pgmap v820: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 783 op/s 2024-04-25T00:17:41.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:40 smithi110 ceph-mon[29187]: pgmap v820: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 783 op/s 2024-04-25T00:17:41.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:40 smithi173 ceph-mon[32402]: pgmap v820: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 783 op/s 2024-04-25T00:17:41.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 875.0 s, 857.8 tps, lat 116.944 ms stddev 22.935 2024-04-25T00:17:43.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:42 smithi188 ceph-mon[33285]: pgmap v821: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-04-25T00:17:43.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:42 smithi110 ceph-mon[29187]: pgmap v821: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-04-25T00:17:43.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:42 smithi173 ceph-mon[32402]: pgmap v821: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-04-25T00:17:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 -- 172.21.15.110:0/3661207314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c101d70 msgr2=0x7fcd0c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 --2- 172.21.15.110:0/3661207314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c101d70 0x7fcd0c10bfe0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009be0 tx=0x7fcd0402a6d0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 -- 172.21.15.110:0/3661207314 shutdown_connections 2024-04-25T00:17:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 --2- 172.21.15.110:0/3661207314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c101d70 0x7fcd0c10bfe0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 --2- 172.21.15.110:0/3661207314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd0c1013b0 0x7fcd0c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 --2- 172.21.15.110:0/3661207314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd0c102d80 0x7fcd0c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 -- 172.21.15.110:0/3661207314 >> 172.21.15.110:0/3661207314 conn(0x7fcd0c0fcd30 msgr2=0x7fcd0c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 -- 172.21.15.110:0/3661207314 shutdown_connections 2024-04-25T00:17:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.810+0000 7fcd11d09640 1 -- 172.21.15.110:0/3661207314 wait complete. 2024-04-25T00:17:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd11d09640 1 Processor -- start 2024-04-25T00:17:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd11d09640 1 -- start start 2024-04-25T00:17:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd11d09640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c1013b0 0x7fcd0c1956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd11d09640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd0c101d70 0x7fcd0c195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd11d09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd0c102d80 0x7fcd0c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd11d09640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c106650 con 0x7fcd0c101d70 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd11d09640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1064d0 con 0x7fcd0c1013b0 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd11d09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1067d0 con 0x7fcd0c102d80 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.811+0000 7fcd0b7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c1013b0 0x7fcd0c1956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0b7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c1013b0 0x7fcd0c1956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:40036/0 (socket says 172.21.15.110:40036) 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0b7fe640 1 -- 172.21.15.110:0/156636630 learned_addr learned my addr 172.21.15.110:0/156636630 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0affd640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd0c101d70 0x7fcd0c195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd0c102d80 0x7fcd0c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0b7fe640 1 -- 172.21.15.110:0/156636630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd0c102d80 msgr2=0x7fcd0c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0b7fe640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd0c102d80 0x7fcd0c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0b7fe640 1 -- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd0c101d70 msgr2=0x7fcd0c195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0b7fe640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd0c101d70 0x7fcd0c195be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0b7fe640 1 -- 172.21.15.110:0/156636630 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c19d270 con 0x7fcd0c1013b0 2024-04-25T00:17:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.812+0000 7fcd0b7fe640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c1013b0 0x7fcd0c1956a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e9f0 tx=0x7fccfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.813+0000 7fcd08ff9640 1 -- 172.21.15.110:0/156636630 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00ccc0 con 0x7fcd0c1013b0 2024-04-25T00:17:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.813+0000 7fcd08ff9640 1 -- 172.21.15.110:0/156636630 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccfc0035f0 con 0x7fcd0c1013b0 2024-04-25T00:17:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.813+0000 7fcd08ff9640 1 -- 172.21.15.110:0/156636630 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00ce60 con 0x7fcd0c1013b0 2024-04-25T00:17:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.813+0000 7fcd11d09640 1 -- 172.21.15.110:0/156636630 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0c19d560 con 0x7fcd0c1013b0 2024-04-25T00:17:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.813+0000 7fcd11d09640 1 -- 172.21.15.110:0/156636630 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0c19da60 con 0x7fcd0c1013b0 2024-04-25T00:17:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd08ff9640 1 -- 172.21.15.110:0/156636630 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fccfc0037a0 con 0x7fcd0c1013b0 2024-04-25T00:17:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd14a11740 1 Processor -- start 2024-04-25T00:17:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd14a11740 1 -- start start 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81c4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81c49b0 0x55b2a81c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd14a11740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b2a8122cb0 con 0x55b2a81bf840 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd14a11740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b2a80cb5b0 con 0x55b2a81c49b0 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd14a11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b2a811ec70 con 0x55b2a81bee50 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.816+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81bf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49712/0 (socket says 172.21.15.110:49712) 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0bfff640 1 -- 172.21.15.110:0/1562345741 learned_addr learned my addr 172.21.15.110:0/1562345741 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0bfff640 1 -- 172.21.15.110:0/1562345741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81c49b0 msgr2=0x55b2a81c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0affd640 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81c49b0 0x55b2a81c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81c49b0 0x55b2a81c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0bfff640 1 -- 172.21.15.110:0/1562345741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 msgr2=0x55b2a81c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0bfff640 1 -- 172.21.15.110:0/1562345741 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2a7e37210 con 0x55b2a81bee50 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd08ff9640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccf40a2cb0 0x7fccf40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0b7fe640 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd0affd640 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81c49b0 0x55b2a81c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.817+0000 7fcd08ff9640 1 -- 172.21.15.110:0/156636630 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fccfc0cf270 con 0x7fcd0c1013b0 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.818+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81bf270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009be0 tx=0x7fcd04030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec08858496f110e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.818+0000 7fcd0b7fe640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccf40a2cb0 0x7fccf40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.818+0000 7fccf1ffb640 1 -- 172.21.15.110:0/1562345741 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04053070 con 0x55b2a81bee50 2024-04-25T00:17:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.818+0000 7fccf1ffb640 1 -- 172.21.15.110:0/1562345741 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd04037040 con 0x55b2a81bee50 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.818+0000 7fccf1ffb640 1 -- 172.21.15.110:0/1562345741 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04036050 con 0x55b2a81bee50 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd14a11740 1 -- 172.21.15.110:0/1562345741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 msgr2=0x55b2a81bf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd0b7fe640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccf40a2cb0 0x7fccf40a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0cd110 tx=0x7fccfc0d2d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd14a11740 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81bf270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009be0 tx=0x7fcd04030430 comp rx=0 tx=0).stop 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd14a11740 1 -- 172.21.15.110:0/1562345741 shutdown_connections 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd14a11740 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81c49b0 0x55b2a81c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd14a11740 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd14a11740 1 --2- 172.21.15.110:0/1562345741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81bf270 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd14a11740 1 -- 172.21.15.110:0/1562345741 >> 172.21.15.110:0/1562345741 conn(0x55b2a81ba4c0 msgr2=0x55b2a81ba8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.820+0000 7fcd14a11740 1 -- 172.21.15.110:0/1562345741 shutdown_connections 2024-04-25T00:17:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.821+0000 7fcd14a11740 1 -- 172.21.15.110:0/1562345741 wait complete. 2024-04-25T00:17:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.821+0000 7fcd14a11740 1 -- start start 2024-04-25T00:17:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.821+0000 7fcd14a11740 1 Processor -- start 2024-04-25T00:17:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.822+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.823+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bf840 0x55b2a81c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.823+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 0x55b2a81d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.823+0000 7fcd14a11740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b2a81c7900 con 0x55b2a81c49b0 2024-04-25T00:17:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.823+0000 7fcd14a11740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b2a80deb50 con 0x55b2a81bf840 2024-04-25T00:17:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.823+0000 7fcd14a11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b2a81c7a80 con 0x55b2a81bee50 2024-04-25T00:17:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.823+0000 7fcd14a11740 10 client.0 _ll_register_callbacks cb 0x55b2a8119ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49718/0 (socket says 172.21.15.110:49718) 2024-04-25T00:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0bfff640 1 -- 172.21.15.110:0/923135411 learned_addr learned my addr 172.21.15.110:0/923135411 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0affd640 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bf840 0x55b2a81c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0bfff640 1 -- 172.21.15.110:0/923135411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bf840 msgr2=0x55b2a81c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bf840 0x55b2a81c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0bfff640 1 -- 172.21.15.110:0/923135411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 msgr2=0x55b2a81d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 0x55b2a81d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.825+0000 7fcd0bfff640 1 -- 172.21.15.110:0/923135411 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2a7f03620 con 0x55b2a81bee50 2024-04-25T00:17:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.826+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81c92a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd04034040 tx=0x7fcd04030dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b07527c2f5501df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.826+0000 7fcceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:17:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.826+0000 7fcceaffd640 1 -- 172.21.15.110:0/923135411 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04053030 con 0x55b2a81bee50 2024-04-25T00:17:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.827+0000 7fcd14a11740 1 -- 172.21.15.110:0/923135411 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd04037040 con 0x55b2a81bee50 2024-04-25T00:17:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.827+0000 7fcceaffd640 1 -- 172.21.15.110:0/923135411 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcd04003db0 con 0x55b2a81bee50 2024-04-25T00:17:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.827+0000 7fcceaffd640 1 -- 172.21.15.110:0/923135411 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcd04004b40 con 0x55b2a81bee50 2024-04-25T00:17:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.829+0000 7fcd14a11740 10 client.26059 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.829+0000 7fcd14a11740 1 -- 172.21.15.110:0/923135411 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2a81c7900 con 0x55b2a81bee50 2024-04-25T00:17:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.829+0000 7fcceaffd640 1 -- 172.21.15.110:0/923135411 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd04004ce0 con 0x55b2a81bee50 2024-04-25T00:17:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.829+0000 7fcceaffd640 1 -- 172.21.15.110:0/923135411 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fcd0404c040 con 0x55b2a81bee50 2024-04-25T00:17:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.831+0000 7fcd14a11740 10 client.26059 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.831+0000 7fcd14a11740 10 client.26059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.831+0000 7fcd14a11740 20 client.26059 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.831+0000 7fcd14a11740 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b2a81d4d40 0x55b2a81d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.831+0000 7fcd14a11740 4 client.26059 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.831+0000 7fcd14a11740 1 -- 172.21.15.110:0/923135411 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcd04037040 con 0x55b2a81d4d40 2024-04-25T00:17:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.831+0000 7fcd0affd640 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b2a81d4d40 0x55b2a81d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.833+0000 7fcd0affd640 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b2a81d4d40 0x55b2a81d7130 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61323558c9e45c server_cookie=27a0201b64b80665 in_seq=0 out_seq=0 2024-04-25T00:17:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.838+0000 7fcceaffd640 10 client.26059 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.838+0000 7fcceaffd640 1 -- 172.21.15.110:0/923135411 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fcd04037040 con 0x55b2a81d4d40 2024-04-25T00:17:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.838+0000 7fcceaffd640 10 client.26059 handle_command_reply: tid=0 2024-04-25T00:17:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.838+0000 7fcd14a11740 1 client.26059 shutdown 2024-04-25T00:17:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.844+0000 7fcd14a11740 1 -- 172.21.15.110:0/923135411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 msgr2=0x55b2a81c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.844+0000 7fcd14a11740 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81c92a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd04034040 tx=0x7fcd04030dd0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.845+0000 7fcd14a11740 1 -- 172.21.15.110:0/923135411 shutdown_connections 2024-04-25T00:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.845+0000 7fcd14a11740 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b2a81d4d40 0x55b2a81d7130 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.845+0000 7fcd14a11740 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 0x55b2a81d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.845+0000 7fcd14a11740 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bf840 0x55b2a81c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.845+0000 7fcd14a11740 1 --2- 172.21.15.110:0/923135411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bee50 0x55b2a81c92a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.845+0000 7fcd14a11740 1 -- 172.21.15.110:0/923135411 >> 172.21.15.110:0/923135411 conn(0x55b2a81ba4c0 msgr2=0x55b2a81b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.848+0000 7fcceaffd640 0 client.26059 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.849+0000 7fcd14a11740 1 -- 172.21.15.110:0/923135411 shutdown_connections 2024-04-25T00:17:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.851+0000 7fcd14a11740 1 -- 172.21.15.110:0/923135411 wait complete. 2024-04-25T00:17:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.851+0000 7fcd14a11740 20 client.26059 trim_cache size 0 max 16384 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.878+0000 7fcd14a11740 1 Processor -- start 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.878+0000 7fcd14a11740 1 -- start start 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bf840 0x55b2a81d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 0x55b2a81bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd14a11740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1a09a0 con 0x55b2a81c49b0 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd14a11740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c19e270 con 0x55b2a81bee50 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd14a11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c071d10 con 0x55b2a81bf840 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bf840 0x55b2a81d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:40066/0 (socket says 172.21.15.110:40066) 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bf840 0x55b2a81d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49722/0 (socket says 172.21.15.110:49722) 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0bfff640 1 -- 172.21.15.110:0/315870752 learned_addr learned my addr 172.21.15.110:0/315870752 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/315870752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 0x55b2a81bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0affd640 1 -- 172.21.15.110:0/315870752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 msgr2=0x55b2a81c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0affd640 1 --2- 172.21.15.110:0/315870752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0affd640 1 -- 172.21.15.110:0/315870752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 msgr2=0x55b2a81bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0affd640 1 --2- 172.21.15.110:0/315870752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 0x55b2a81bdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0affd640 1 -- 172.21.15.110:0/315870752 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2a81bb230 con 0x55b2a81bf840 2024-04-25T00:17:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.879+0000 7fcd0affd640 1 --2- 172.21.15.110:0/315870752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bf840 0x55b2a81d07e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55b2a81d7900 tx=0x7fcd00005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b25c94dcef08f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.880+0000 7fccf17fa640 1 -- 172.21.15.110:0/315870752 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000206c0 con 0x55b2a81bf840 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.880+0000 7fccf17fa640 1 -- 172.21.15.110:0/315870752 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd0002c070 con 0x55b2a81bf840 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.880+0000 7fccf17fa640 1 -- 172.21.15.110:0/315870752 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0001fb70 con 0x55b2a81bf840 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 -- 172.21.15.110:0/315870752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bf840 msgr2=0x55b2a81d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 --2- 172.21.15.110:0/315870752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bf840 0x55b2a81d07e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55b2a81d7900 tx=0x7fcd00005f90 comp rx=0 tx=0).stop 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 -- 172.21.15.110:0/315870752 shutdown_connections 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 --2- 172.21.15.110:0/315870752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81c49b0 0x55b2a81bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 --2- 172.21.15.110:0/315870752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81bf840 0x55b2a81d07e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 --2- 172.21.15.110:0/315870752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 -- 172.21.15.110:0/315870752 >> 172.21.15.110:0/315870752 conn(0x55b2a81ba4c0 msgr2=0x55b2a81bd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 -- 172.21.15.110:0/315870752 shutdown_connections 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.881+0000 7fcd14a11740 1 -- 172.21.15.110:0/315870752 wait complete. 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 1 -- start start 2024-04-25T00:17:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 1 Processor -- start 2024-04-25T00:17:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81c49b0 0x55b2a81cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c072380 con 0x55b2a81bf840 2024-04-25T00:17:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c0737f0 con 0x55b2a81bee50 2024-04-25T00:17:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c072a50 con 0x55b2a81c49b0 2024-04-25T00:17:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd14a11740 10 client.0 _ll_register_callbacks cb 0x7fccfc0d2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd0affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.882+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81c49b0 0x55b2a81cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81c49b0 0x55b2a81cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:49728/0 (socket says 172.21.15.110:49728) 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0bfff640 1 -- 172.21.15.110:0/2201791135 learned_addr learned my addr 172.21.15.110:0/2201791135 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0affd640 1 -- 172.21.15.110:0/2201791135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81c49b0 msgr2=0x55b2a81cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0affd640 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81c49b0 0x55b2a81cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0affd640 1 -- 172.21.15.110:0/2201791135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 msgr2=0x55b2a81cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0affd640 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0affd640 1 -- 172.21.15.110:0/2201791135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2a81cf470 con 0x55b2a81bf840 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81c49b0 0x55b2a81cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd0affd640 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81cdec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55b2a81bab50 tx=0x7fcd0001f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f249676b9927b23c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcceaffd640 1 -- 172.21.15.110:0/2201791135 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00034070 con 0x55b2a81bf840 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcceaffd640 1 -- 172.21.15.110:0/2201791135 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fcd0001fcc0 con 0x55b2a81bf840 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.883+0000 7fcd14a11740 1 -- 172.21.15.110:0/2201791135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd0002c070 con 0x55b2a81bf840 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.884+0000 7fcceaffd640 1 -- 172.21.15.110:0/2201791135 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcd00040050 con 0x55b2a81bf840 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.884+0000 7fcd14a11740 10 client.16482 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.884+0000 7fcd14a11740 1 -- 172.21.15.110:0/2201791135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2a81d0780 con 0x55b2a81bf840 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.884+0000 7fcceaffd640 1 -- 172.21.15.110:0/2201791135 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd00005500 con 0x55b2a81bf840 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.884+0000 7fcceaffd640 1 -- 172.21.15.110:0/2201791135 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fcd000204c0 con 0x55b2a81bf840 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcd14a11740 10 client.16482 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcd14a11740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcd14a11740 20 client.16482 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcd14a11740 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b2a823b480 0x55b2a823d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcd14a11740 4 client.16482 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:17:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcd14a11740 1 -- 172.21.15.110:0/2201791135 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fcd0002c070 con 0x55b2a823b480 2024-04-25T00:17:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b2a823b480 0x55b2a823d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcd0bfff640 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b2a823b480 0x55b2a823d8a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b65c0769ce6f591 server_cookie=6dc601240dd37940 in_seq=0 out_seq=0 2024-04-25T00:17:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.885+0000 7fcceaffd640 10 client.16482 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.904+0000 7fcceaffd640 1 -- 172.21.15.110:0/2201791135 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fcd0002c070 con 0x55b2a823b480 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.904+0000 7fcceaffd640 10 client.16482 handle_command_reply: tid=0 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.904+0000 7fcd14a11740 1 client.16482 shutdown 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.904+0000 7fcd14a11740 1 -- 172.21.15.110:0/2201791135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 msgr2=0x55b2a81cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.904+0000 7fcd14a11740 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81cdec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55b2a81bab50 tx=0x7fcd0001f8d0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 1 -- 172.21.15.110:0/2201791135 shutdown_connections 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b2a823b480 0x55b2a823d8a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcceaffd640 0 client.16482 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b2a81c49b0 0x55b2a81cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2a81bf840 0x55b2a81cdec0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 1 --2- 172.21.15.110:0/2201791135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b2a81bee50 0x55b2a81cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 1 -- 172.21.15.110:0/2201791135 >> 172.21.15.110:0/2201791135 conn(0x55b2a81ba4c0 msgr2=0x55b2a81c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 1 -- 172.21.15.110:0/2201791135 shutdown_connections 2024-04-25T00:17:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 1 -- 172.21.15.110:0/2201791135 wait complete. 2024-04-25T00:17:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.905+0000 7fcd14a11740 20 client.16482 trim_cache size 0 max 16384 2024-04-25T00:17:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:17:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:17:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:17:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:17:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 -- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccf40a2cb0 msgr2=0x7fccf40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccf40a2cb0 0x7fccf40a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0cd110 tx=0x7fccfc0d2d50 comp rx=0 tx=0).stop 2024-04-25T00:17:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 -- 172.21.15.110:0/156636630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c1013b0 msgr2=0x7fcd0c1956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c1013b0 0x7fcd0c1956a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e9f0 tx=0x7fccfc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 -- 172.21.15.110:0/156636630 shutdown_connections 2024-04-25T00:17:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fccf40a2cb0 0x7fccf40a5170 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd0c102d80 0x7fcd0c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd0c101d70 0x7fcd0c195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 --2- 172.21.15.110:0/156636630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c1013b0 0x7fcd0c1956a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 -- 172.21.15.110:0/156636630 >> 172.21.15.110:0/156636630 conn(0x7fcd0c0fcd30 msgr2=0x7fcd0c106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 -- 172.21.15.110:0/156636630 shutdown_connections 2024-04-25T00:17:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:44.906+0000 7fcd11d09640 1 -- 172.21.15.110:0/156636630 wait complete. 2024-04-25T00:17:45.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:44 smithi188 ceph-mon[33285]: pgmap v822: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-04-25T00:17:45.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:44 smithi110 ceph-mon[29187]: pgmap v822: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-04-25T00:17:45.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:44 smithi173 ceph-mon[32402]: pgmap v822: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-04-25T00:17:46.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 880.0 s, 814.0 tps, lat 122.147 ms stddev 21.333 2024-04-25T00:17:47.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:46 smithi188 ceph-mon[33285]: pgmap v823: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 844 op/s 2024-04-25T00:17:47.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:46 smithi110 ceph-mon[29187]: pgmap v823: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 844 op/s 2024-04-25T00:17:47.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:46 smithi173 ceph-mon[32402]: pgmap v823: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 844 op/s 2024-04-25T00:17:49.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:48 smithi188 ceph-mon[33285]: pgmap v824: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 976 op/s 2024-04-25T00:17:49.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:48 smithi110 ceph-mon[29187]: pgmap v824: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 976 op/s 2024-04-25T00:17:49.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:48 smithi173 ceph-mon[32402]: pgmap v824: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 976 op/s 2024-04-25T00:17:51.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:50 smithi188 ceph-mon[33285]: pgmap v825: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 799 op/s 2024-04-25T00:17:51.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:50 smithi110 ceph-mon[29187]: pgmap v825: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 799 op/s 2024-04-25T00:17:51.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:50 smithi173 ceph-mon[32402]: pgmap v825: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 799 op/s 2024-04-25T00:17:51.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 885.0 s, 848.8 tps, lat 118.674 ms stddev 18.688 2024-04-25T00:17:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:52 smithi188 ceph-mon[33285]: pgmap v826: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-04-25T00:17:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:52 smithi110 ceph-mon[29187]: pgmap v826: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-04-25T00:17:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:52 smithi173 ceph-mon[32402]: pgmap v826: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-04-25T00:17:55.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:54 smithi188 ceph-mon[33285]: pgmap v827: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 790 op/s 2024-04-25T00:17:55.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:54 smithi110 ceph-mon[29187]: pgmap v827: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 790 op/s 2024-04-25T00:17:55.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:54 smithi173 ceph-mon[32402]: pgmap v827: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 790 op/s 2024-04-25T00:17:56.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:55 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:56.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:55 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:56.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:55 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:56.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 890.0 s, 851.6 tps, lat 116.798 ms stddev 20.454 2024-04-25T00:17:57.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:57.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:57.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:56 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:17:57.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:56 smithi188 ceph-mon[33285]: pgmap v828: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-04-25T00:17:57.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:57.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:57.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:56 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:17:57.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:56 smithi110 ceph-mon[29187]: pgmap v828: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-04-25T00:17:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:56 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:17:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:56 smithi173 ceph-mon[32402]: pgmap v828: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-04-25T00:17:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 -- 172.21.15.110:0/2970587882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff634104c30 msgr2=0x7ff634107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 --2- 172.21.15.110:0/2970587882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff634104c30 0x7ff634107020 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff628009900 tx=0x7ff62802a410 comp rx=0 tx=0).stop 2024-04-25T00:17:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 -- 172.21.15.110:0/2970587882 shutdown_connections 2024-04-25T00:17:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 --2- 172.21.15.110:0/2970587882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634107560 0x7ff634109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 --2- 172.21.15.110:0/2970587882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff634104c30 0x7ff634107020 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 --2- 172.21.15.110:0/2970587882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff634102300 0x7ff6341046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 -- 172.21.15.110:0/2970587882 >> 172.21.15.110:0/2970587882 conn(0x7ff6340fbc50 msgr2=0x7ff6340fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 -- 172.21.15.110:0/2970587882 shutdown_connections 2024-04-25T00:17:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.037+0000 7ff63961c640 1 -- 172.21.15.110:0/2970587882 wait complete. 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.038+0000 7ff63961c640 1 Processor -- start 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.038+0000 7ff63961c640 1 -- start start 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.038+0000 7ff63961c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff634102300 0x7ff634101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.038+0000 7ff63961c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff634104c30 0x7ff6341002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff63961c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634107560 0x7ff6341007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff63961c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff63410d300 con 0x7ff634102300 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff63961c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff63410d180 con 0x7ff634104c30 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6327fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff634104c30 0x7ff6341002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff634102300 0x7ff634101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6327fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff634104c30 0x7ff6341002b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43016/0 (socket says 172.21.15.110:43016) 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6327fc640 1 -- 172.21.15.110:0/3050404705 learned_addr learned my addr 172.21.15.110:0/3050404705 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff63961c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff63410d480 con 0x7ff634107560 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6337fe640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634107560 0x7ff6341007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6327fc640 1 -- 172.21.15.110:0/3050404705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634107560 msgr2=0x7ff6341007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6327fc640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634107560 0x7ff6341007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6327fc640 1 -- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff634102300 msgr2=0x7ff634101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6327fc640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff634102300 0x7ff634101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.039+0000 7ff6327fc640 1 -- 172.21.15.110:0/3050404705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff634101080 con 0x7ff634104c30 2024-04-25T00:17:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.040+0000 7ff6327fc640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff634104c30 0x7ff6341002b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6280314e0 tx=0x7ff628004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.040+0000 7ff61bfff640 1 -- 172.21.15.110:0/3050404705 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62803f030 con 0x7ff634104c30 2024-04-25T00:17:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.040+0000 7ff63961c640 1 -- 172.21.15.110:0/3050404705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6341a1a70 con 0x7ff634104c30 2024-04-25T00:17:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.040+0000 7ff61bfff640 1 -- 172.21.15.110:0/3050404705 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff628003aa0 con 0x7ff634104c30 2024-04-25T00:17:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.040+0000 7ff63961c640 1 -- 172.21.15.110:0/3050404705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6341a1fc0 con 0x7ff634104c30 2024-04-25T00:17:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.040+0000 7ff61bfff640 1 -- 172.21.15.110:0/3050404705 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628035070 con 0x7ff634104c30 2024-04-25T00:17:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.043+0000 7ff63c40a740 1 Processor -- start 2024-04-25T00:17:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.043+0000 7ff63c40a740 1 -- start start 2024-04-25T00:17:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb15a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb1ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb1d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff63c40a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b6cfa70cb0 con 0x55b6cfb16050 2024-04-25T00:17:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff63c40a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b6cfa195b0 con 0x55b6cfb15660 2024-04-25T00:17:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff63c40a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b6cfa6cc70 con 0x55b6cfb1b1c0 2024-04-25T00:17:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff61bfff640 1 -- 172.21.15.110:0/3050404705 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff628042030 con 0x7ff634104c30 2024-04-25T00:17:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb15a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb15a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43028/0 (socket says 172.21.15.110:43028) 2024-04-25T00:17:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6337fe640 1 -- 172.21.15.110:0/1257224064 learned_addr learned my addr 172.21.15.110:0/1257224064 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6327fc640 1 --2- 172.21.15.110:0/1257224064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb1d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6337fe640 1 -- 172.21.15.110:0/1257224064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb1b1c0 msgr2=0x55b6cfb1d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6337fe640 1 --2- 172.21.15.110:0/1257224064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb1d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6337fe640 1 -- 172.21.15.110:0/1257224064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 msgr2=0x55b6cfb1ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6337fe640 1 --2- 172.21.15.110:0/1257224064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb1ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.044+0000 7ff6337fe640 1 -- 172.21.15.110:0/1257224064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6cf785210 con 0x55b6cfb15660 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.045+0000 7ff6337fe640 1 --2- 172.21.15.110:0/1257224064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb15a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009fd0 tx=0x7ff62c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f42d97e9ebba40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.045+0000 7ff6197fa640 1 -- 172.21.15.110:0/1257224064 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c004d20 con 0x55b6cfb15660 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.045+0000 7ff6197fa640 1 -- 172.21.15.110:0/1257224064 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff62c004050 con 0x55b6cfb15660 2024-04-25T00:17:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.045+0000 7ff6197fa640 1 -- 172.21.15.110:0/1257224064 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c005280 con 0x55b6cfb15660 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.045+0000 7ff61bfff640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff61c0a2cb0 0x7ff61c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.045+0000 7ff61bfff640 1 -- 172.21.15.110:0/3050404705 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff6280ef770 con 0x7ff634104c30 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff632ffd640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff61c0a2cb0 0x7ff61c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 -- 172.21.15.110:0/1257224064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 msgr2=0x55b6cfb15a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1257224064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb15a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009fd0 tx=0x7ff62c0118d0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 -- 172.21.15.110:0/1257224064 shutdown_connections 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1257224064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb1d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1257224064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb1ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1257224064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb15a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 -- 172.21.15.110:0/1257224064 >> 172.21.15.110:0/1257224064 conn(0x55b6cfb084c0 msgr2=0x55b6cfb088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 -- 172.21.15.110:0/1257224064 shutdown_connections 2024-04-25T00:17:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.046+0000 7ff63c40a740 1 -- 172.21.15.110:0/1257224064 wait complete. 2024-04-25T00:17:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.047+0000 7ff632ffd640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff61c0a2cb0 0x7ff61c0a5170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55b6cfb16450 tx=0x7ff624009840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.047+0000 7ff63c40a740 1 -- start start 2024-04-25T00:17:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.047+0000 7ff63c40a740 1 Processor -- start 2024-04-25T00:17:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb0e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb0eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb13ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff63c40a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b6cfb0cf50 con 0x55b6cfb1b1c0 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff63c40a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b6cfa2cb50 con 0x55b6cfb15660 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff63c40a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b6cfb0d0d0 con 0x55b6cfb16050 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb0e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb0e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:43034/0 (socket says 172.21.15.110:43034) 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff632ffd640 1 -- 172.21.15.110:0/797993264 learned_addr learned my addr 172.21.15.110:0/797993264 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff6327fc640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb0eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff6337fe640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb13ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff63c40a740 10 client.0 _ll_register_callbacks cb 0x55b6cfa67ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff632ffd640 1 -- 172.21.15.110:0/797993264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 msgr2=0x55b6cfb0eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff632ffd640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb0eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff632ffd640 1 -- 172.21.15.110:0/797993264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 msgr2=0x55b6cfb13ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff632ffd640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb13ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff632ffd640 1 -- 172.21.15.110:0/797993264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6cf851620 con 0x55b6cfb15660 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff6327fc640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb0eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff632ffd640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb0e980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff62401cb80 tx=0x7ff624005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a52e86a88f946a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff616ff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.048+0000 7ff6337fe640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb13ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff616ff5640 1 -- 172.21.15.110:0/797993264 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624009e60 con 0x55b6cfb15660 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff63c40a740 1 -- 172.21.15.110:0/797993264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff62c004050 con 0x55b6cfb15660 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff616ff5640 1 -- 172.21.15.110:0/797993264 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff62400fb40 con 0x55b6cfb15660 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff616ff5640 1 -- 172.21.15.110:0/797993264 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff62400fd40 con 0x55b6cfb15660 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff63c40a740 10 client.26081 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff63c40a740 1 -- 172.21.15.110:0/797993264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6cfb32cd0 con 0x55b6cfb15660 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff616ff5640 1 -- 172.21.15.110:0/797993264 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff62c004050 con 0x55b6cfb15660 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.049+0000 7ff616ff5640 1 -- 172.21.15.110:0/797993264 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7ff62401d410 con 0x55b6cfb15660 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff63c40a740 10 client.26081 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff63c40a740 10 client.26081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff63c40a740 20 client.26081 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff63c40a740 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b6cfb34880 0x55b6cfb36d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff63c40a740 4 client.26081 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff63c40a740 1 -- 172.21.15.110:0/797993264 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6cfa7e1f0 con 0x55b6cfb34880 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff6327fc640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b6cfb34880 0x55b6cfb36d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff616ff5640 10 client.26081 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.050+0000 7ff6327fc640 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b6cfb34880 0x55b6cfb36d20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36eecf971543024b server_cookie=ab145dcc5fe57f4a in_seq=0 out_seq=0 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff616ff5640 1 -- 172.21.15.110:0/797993264 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6cfa7e1f0 con 0x55b6cfb34880 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff616ff5640 10 client.26081 handle_command_reply: tid=0 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 client.26081 shutdown 2024-04-25T00:17:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 -- 172.21.15.110:0/797993264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 msgr2=0x55b6cfb0e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb0e980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff62401cb80 tx=0x7ff624005f90 comp rx=0 tx=0).stop 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 -- 172.21.15.110:0/797993264 shutdown_connections 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b6cfb34880 0x55b6cfb36d20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb13ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb0eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 --2- 172.21.15.110:0/797993264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb0e980 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff616ff5640 0 client.26081 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 -- 172.21.15.110:0/797993264 >> 172.21.15.110:0/797993264 conn(0x55b6cfb084c0 msgr2=0x55b6cfb075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 -- 172.21.15.110:0/797993264 shutdown_connections 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 1 -- 172.21.15.110:0/797993264 wait complete. 2024-04-25T00:17:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.051+0000 7ff63c40a740 20 client.26081 trim_cache size 0 max 16384 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff63c40a740 1 Processor -- start 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff63c40a740 1 -- start start 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb15660 0x55b6cfb17550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb0b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff63c40a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a90d0 con 0x55b6cfb16050 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff63c40a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a69a0 con 0x55b6cfb1b1c0 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff63c40a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a47f0 con 0x55b6cfb15660 2024-04-25T00:17:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6327fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6327fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb17a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38712/0 (socket says 172.21.15.110:38712) 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6327fc640 1 -- 172.21.15.110:0/2190661764 learned_addr learned my addr 172.21.15.110:0/2190661764 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb15660 0x55b6cfb17550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6337fe640 1 --2- 172.21.15.110:0/2190661764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb0b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6327fc640 1 -- 172.21.15.110:0/2190661764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb15660 msgr2=0x55b6cfb17550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6327fc640 1 --2- 172.21.15.110:0/2190661764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb15660 0x55b6cfb17550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6327fc640 1 -- 172.21.15.110:0/2190661764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb1b1c0 msgr2=0x55b6cfb0b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6327fc640 1 --2- 172.21.15.110:0/2190661764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb0b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.079+0000 7ff6327fc640 1 -- 172.21.15.110:0/2190661764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6cfb1a840 con 0x55b6cfb16050 2024-04-25T00:17:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.080+0000 7ff6327fc640 1 --2- 172.21.15.110:0/2190661764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb17a90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff62803b580 tx=0x7ff6280dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f0cd3d1ea2e441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.080+0000 7ff618ff9640 1 -- 172.21.15.110:0/2190661764 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628049040 con 0x55b6cfb16050 2024-04-25T00:17:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.080+0000 7ff618ff9640 1 -- 172.21.15.110:0/2190661764 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6280bc050 con 0x55b6cfb16050 2024-04-25T00:17:58.083 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.080+0000 7ff618ff9640 1 -- 172.21.15.110:0/2190661764 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62803bd20 con 0x55b6cfb16050 2024-04-25T00:17:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 -- 172.21.15.110:0/2190661764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 msgr2=0x55b6cfb17a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 --2- 172.21.15.110:0/2190661764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb17a90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff62803b580 tx=0x7ff6280dc040 comp rx=0 tx=0).stop 2024-04-25T00:17:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 -- 172.21.15.110:0/2190661764 shutdown_connections 2024-04-25T00:17:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 --2- 172.21.15.110:0/2190661764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb0b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 --2- 172.21.15.110:0/2190661764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb16050 0x55b6cfb17a90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 --2- 172.21.15.110:0/2190661764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb15660 0x55b6cfb17550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 -- 172.21.15.110:0/2190661764 >> 172.21.15.110:0/2190661764 conn(0x55b6cfb084c0 msgr2=0x55b6cfb0d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 -- 172.21.15.110:0/2190661764 shutdown_connections 2024-04-25T00:17:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.081+0000 7ff63c40a740 1 -- 172.21.15.110:0/2190661764 wait complete. 2024-04-25T00:17:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 1 -- start start 2024-04-25T00:17:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 1 Processor -- start 2024-04-25T00:17:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb45110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb45650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb98970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a4e60 con 0x55b6cfb1b1c0 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a62d0 con 0x55b6cfb15660 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a5530 con 0x55b6cfb16050 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff6327fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb45650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb45110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff6327fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb45650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:48636/0 (socket says 172.21.15.110:48636) 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff6327fc640 1 -- 172.21.15.110:0/1065901785 learned_addr learned my addr 172.21.15.110:0/1065901785 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff6337fe640 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb98970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.082+0000 7ff63c40a740 10 client.0 _ll_register_callbacks cb 0x7ff624009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff6327fc640 1 -- 172.21.15.110:0/1065901785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 msgr2=0x55b6cfb45110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff6327fc640 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb45110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff6327fc640 1 -- 172.21.15.110:0/1065901785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 msgr2=0x55b6cfb98970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff6327fc640 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb98970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff6327fc640 1 -- 172.21.15.110:0/1065901785 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6cfb98ee0 con 0x55b6cfb16050 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff632ffd640 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb45110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff6327fc640 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb45650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff6280bbd50 tx=0x7ff628009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d01b01b979b3d213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff616ff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff616ff5640 1 -- 172.21.15.110:0/1065901785 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628049050 con 0x55b6cfb16050 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff616ff5640 1 -- 172.21.15.110:0/1065901785 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff6280b86c0 con 0x55b6cfb16050 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff63c40a740 1 -- 172.21.15.110:0/1065901785 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6280bc050 con 0x55b6cfb16050 2024-04-25T00:17:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff616ff5640 1 -- 172.21.15.110:0/1065901785 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff62809f050 con 0x55b6cfb16050 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff63c40a740 10 client.26083 fetch_fsmap learned FSMap version 32 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff63c40a740 1 -- 172.21.15.110:0/1065901785 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6cfb9a1d0 con 0x55b6cfb16050 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.083+0000 7ff616ff5640 1 -- 172.21.15.110:0/1065901785 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6280c0350 con 0x55b6cfb16050 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff616ff5640 1 -- 172.21.15.110:0/1065901785 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7ff6280bc2b0 con 0x55b6cfb16050 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff63c40a740 10 client.26083 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff63c40a740 10 client.26083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff63c40a740 20 client.26083 populate_metadata read hostname 'smithi110' 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b6cfb9a9c0 0x55b6cfb9cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff63c40a740 4 client.26083 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:17:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff63c40a740 1 -- 172.21.15.110:0/1065901785 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff6280bc050 con 0x55b6cfb9a9c0 2024-04-25T00:17:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff632ffd640 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b6cfb9a9c0 0x55b6cfb9cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff616ff5640 10 client.26083 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.084+0000 7ff632ffd640 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b6cfb9a9c0 0x55b6cfb9cde0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a5344649819b39 server_cookie=a175744446751a76 in_seq=0 out_seq=0 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.100+0000 7ff616ff5640 1 -- 172.21.15.110:0/1065901785 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff6280bc050 con 0x55b6cfb9a9c0 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.100+0000 7ff616ff5640 10 client.26083 handle_command_reply: tid=0 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 client.26083 shutdown 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 -- 172.21.15.110:0/1065901785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 msgr2=0x55b6cfb45650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb45650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff6280bbd50 tx=0x7ff628009560 comp rx=0 tx=0).stop 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 -- 172.21.15.110:0/1065901785 shutdown_connections 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55b6cfb9a9c0 0x55b6cfb9cde0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff616ff5640 0 client.26083 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6cfb1b1c0 0x55b6cfb98970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6cfb16050 0x55b6cfb45650 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 --2- 172.21.15.110:0/1065901785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6cfb15660 0x55b6cfb45110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 -- 172.21.15.110:0/1065901785 >> 172.21.15.110:0/1065901785 conn(0x55b6cfb084c0 msgr2=0x55b6cfb0d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 -- 172.21.15.110:0/1065901785 shutdown_connections 2024-04-25T00:17:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 1 -- 172.21.15.110:0/1065901785 wait complete. 2024-04-25T00:17:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.101+0000 7ff63c40a740 20 client.26083 trim_cache size 0 max 16384 2024-04-25T00:17:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:17:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:17:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:17:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:17:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 -- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff61c0a2cb0 msgr2=0x7ff61c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff61c0a2cb0 0x7ff61c0a5170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55b6cfb16450 tx=0x7ff624009840 comp rx=0 tx=0).stop 2024-04-25T00:17:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 -- 172.21.15.110:0/3050404705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff634104c30 msgr2=0x7ff6341002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff634104c30 0x7ff6341002b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6280314e0 tx=0x7ff628004270 comp rx=0 tx=0).stop 2024-04-25T00:17:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 -- 172.21.15.110:0/3050404705 shutdown_connections 2024-04-25T00:17:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff61c0a2cb0 0x7ff61c0a5170 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff634107560 0x7ff6341007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff634104c30 0x7ff6341002b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 --2- 172.21.15.110:0/3050404705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff634102300 0x7ff634101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 -- 172.21.15.110:0/3050404705 >> 172.21.15.110:0/3050404705 conn(0x7ff6340fbc50 msgr2=0x7ff634102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.102+0000 7ff63961c640 1 -- 172.21.15.110:0/3050404705 shutdown_connections 2024-04-25T00:17:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:17:58.103+0000 7ff63961c640 1 -- 172.21.15.110:0/3050404705 wait complete. 2024-04-25T00:17:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:17:58 smithi188 ceph-mon[33285]: pgmap v829: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 984 op/s 2024-04-25T00:17:59.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:17:58 smithi110 ceph-mon[29187]: pgmap v829: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 984 op/s 2024-04-25T00:17:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:17:58 smithi173 ceph-mon[32402]: pgmap v829: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 984 op/s 2024-04-25T00:18:01.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:00 smithi188 ceph-mon[33285]: pgmap v830: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 838 op/s 2024-04-25T00:18:01.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:00 smithi110 ceph-mon[29187]: pgmap v830: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 838 op/s 2024-04-25T00:18:01.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:00 smithi173 ceph-mon[32402]: pgmap v830: 97 pgs: 97 active+clean; 1.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 838 op/s 2024-04-25T00:18:01.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 895.0 s, 829.4 tps, lat 120.719 ms stddev 16.899 2024-04-25T00:18:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:02 smithi188 ceph-mon[33285]: pgmap v831: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-04-25T00:18:03.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:02 smithi110 ceph-mon[29187]: pgmap v831: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-04-25T00:18:03.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:02 smithi173 ceph-mon[32402]: pgmap v831: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-04-25T00:18:05.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:04 smithi188 ceph-mon[33285]: pgmap v832: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-04-25T00:18:05.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:04 smithi110 ceph-mon[29187]: pgmap v832: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-04-25T00:18:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:04 smithi173 ceph-mon[32402]: pgmap v832: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-04-25T00:18:06.589 INFO:teuthology.orchestra.run.smithi110.stderr:progress: 900.0 s, 826.6 tps, lat 121.247 ms stddev 19.687 2024-04-25T00:18:06.751 INFO:teuthology.orchestra.run.smithi110.stdout:transaction type: 2024-04-25T00:18:06.751 INFO:teuthology.orchestra.run.smithi110.stdout:scaling factor: 32 2024-04-25T00:18:06.751 INFO:teuthology.orchestra.run.smithi110.stdout:query mode: simple 2024-04-25T00:18:06.751 INFO:teuthology.orchestra.run.smithi110.stdout:number of clients: 100 2024-04-25T00:18:06.751 INFO:teuthology.orchestra.run.smithi110.stdout:number of threads: 4 2024-04-25T00:18:06.751 INFO:teuthology.orchestra.run.smithi110.stdout:duration: 900 s 2024-04-25T00:18:06.751 INFO:teuthology.orchestra.run.smithi110.stdout:number of transactions actually processed: 795053 2024-04-25T00:18:06.751 INFO:teuthology.orchestra.run.smithi110.stdout:latency average = 113.172 ms 2024-04-25T00:18:06.752 INFO:teuthology.orchestra.run.smithi110.stdout:latency stddev = 68.305 ms 2024-04-25T00:18:06.752 INFO:teuthology.orchestra.run.smithi110.stdout:tps = 883.234319 (including connections establishing) 2024-04-25T00:18:06.752 INFO:teuthology.orchestra.run.smithi110.stdout:tps = 883.244424 (excluding connections establishing) 2024-04-25T00:18:06.824 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-04-25T00:18:07.011 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:06 smithi110 ceph-mon[29187]: pgmap v833: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 839 op/s 2024-04-25T00:18:07.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:06 smithi188 ceph-mon[33285]: pgmap v833: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 839 op/s 2024-04-25T00:18:07.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:06 smithi173 ceph-mon[32402]: pgmap v833: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 11 MiB/s wr, 839 op/s 2024-04-25T00:18:09.155 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:08 smithi173 ceph-mon[32402]: pgmap v834: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-04-25T00:18:09.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:08 smithi188 ceph-mon[33285]: pgmap v834: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-04-25T00:18:09.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:08 smithi110 ceph-mon[29187]: pgmap v834: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-04-25T00:18:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.260+0000 7f0c6a678640 1 -- 172.21.15.110:0/467122130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c64076100 msgr2=0x7f0c64076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.260+0000 7f0c6a678640 1 --2- 172.21.15.110:0/467122130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c64076100 0x7f0c64076520 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009900 tx=0x7f0c6002a440 comp rx=0 tx=0).stop 2024-04-25T00:18:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.262+0000 7f0c6a678640 1 -- 172.21.15.110:0/467122130 shutdown_connections 2024-04-25T00:18:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.262+0000 7f0c6a678640 1 --2- 172.21.15.110:0/467122130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c64077ce0 0x7f0c64078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.262+0000 7f0c6a678640 1 --2- 172.21.15.110:0/467122130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64077340 0x7f0c640777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.262+0000 7f0c6a678640 1 --2- 172.21.15.110:0/467122130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c64076100 0x7f0c64076520 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009900 tx=0x7f0c6002a440 comp rx=0 tx=0).stop 2024-04-25T00:18:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.262+0000 7f0c6a678640 1 -- 172.21.15.110:0/467122130 >> 172.21.15.110:0/467122130 conn(0x7f0c640fed70 msgr2=0x7f0c641011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:10 smithi188 ceph-mon[33285]: pgmap v835: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 825 op/s 2024-04-25T00:18:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.267+0000 7f0c6a678640 1 -- 172.21.15.110:0/467122130 shutdown_connections 2024-04-25T00:18:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.269+0000 7f0c6a678640 1 -- 172.21.15.110:0/467122130 wait complete. 2024-04-25T00:18:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.270+0000 7f0c6a678640 1 Processor -- start 2024-04-25T00:18:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.271+0000 7f0c6a678640 1 -- start start 2024-04-25T00:18:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.271+0000 7f0c6a678640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c64077340 0x7f0c6419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.271+0000 7f0c6a678640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64077ce0 0x7f0c6419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.271+0000 7f0c6a678640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c641a37d0 0x7f0c641a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.271+0000 7f0c6a678640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64113890 con 0x7f0c641a37d0 2024-04-25T00:18:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.271+0000 7f0c6a678640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64113710 con 0x7f0c64077ce0 2024-04-25T00:18:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.271+0000 7f0c6a678640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64113a10 con 0x7f0c64077340 2024-04-25T00:18:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.273+0000 7f0c69676640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c64077340 0x7f0c6419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.273+0000 7f0c69e77640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c641a37d0 0x7f0c641a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.273+0000 7f0c69e77640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c641a37d0 0x7f0c641a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59878/0 (socket says 172.21.15.110:59878) 2024-04-25T00:18:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.273+0000 7f0c69e77640 1 -- 172.21.15.110:0/2167773652 learned_addr learned my addr 172.21.15.110:0/2167773652 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.273+0000 7f0c68e75640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64077ce0 0x7f0c6419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.278+0000 7f0c68e75640 1 -- 172.21.15.110:0/2167773652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c64077340 msgr2=0x7f0c6419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.278+0000 7f0c68e75640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c64077340 0x7f0c6419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.278+0000 7f0c68e75640 1 -- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c641a37d0 msgr2=0x7f0c641a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.278+0000 7f0c68e75640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c641a37d0 0x7f0c641a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.278+0000 7f0c68e75640 1 -- 172.21.15.110:0/2167773652 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c64103360 con 0x7f0c64077ce0 2024-04-25T00:18:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.279+0000 7f0c68e75640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64077ce0 0x7f0c6419e770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c00bd50 tx=0x7f0c5c00be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:10 smithi110 ceph-mon[29187]: pgmap v835: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 825 op/s 2024-04-25T00:18:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.286+0000 7f0c5a7fc640 1 -- 172.21.15.110:0/2167773652 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c019070 con 0x7f0c64077ce0 2024-04-25T00:18:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.286+0000 7f0c6a678640 1 -- 172.21.15.110:0/2167773652 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c64103650 con 0x7f0c64077ce0 2024-04-25T00:18:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.286+0000 7f0c6a678640 1 -- 172.21.15.110:0/2167773652 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c64103c20 con 0x7f0c64077ce0 2024-04-25T00:18:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.286+0000 7f0c5a7fc640 1 -- 172.21.15.110:0/2167773652 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c5c0148f0 con 0x7f0c64077ce0 2024-04-25T00:18:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.286+0000 7f0c5a7fc640 1 -- 172.21.15.110:0/2167773652 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c012910 con 0x7f0c64077ce0 2024-04-25T00:18:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.289+0000 7f0c6e5ee740 1 Processor -- start 2024-04-25T00:18:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.291+0000 7f0c6e5ee740 1 -- start start 2024-04-25T00:18:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.291+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6dd93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.291+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd99b0 0x55f9d6dde4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.291+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6de0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.291+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f9d6d3ccb0 con 0x55f9d6ddeb20 2024-04-25T00:18:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.291+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f9d6ce55b0 con 0x55f9d6dd99b0 2024-04-25T00:18:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.291+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f9d6d38c70 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.295+0000 7f0c69e77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6dd93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.295+0000 7f0c69e77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6dd93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:41616/0 (socket says 172.21.15.110:41616) 2024-04-25T00:18:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.295+0000 7f0c69e77640 1 -- 172.21.15.110:0/345841976 learned_addr learned my addr 172.21.15.110:0/345841976 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:11.299 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.296+0000 7f0c68e75640 1 --2- 172.21.15.110:0/345841976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6de0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.296+0000 7f0c68e75640 1 -- 172.21.15.110:0/345841976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd8fc0 msgr2=0x55f9d6dd93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.296+0000 7f0c68e75640 1 --2- 172.21.15.110:0/345841976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6dd93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.296+0000 7f0c68e75640 1 -- 172.21.15.110:0/345841976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd99b0 msgr2=0x55f9d6dde4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:18:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.296+0000 7f0c68e75640 1 --2- 172.21.15.110:0/345841976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd99b0 0x55f9d6dde4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.296+0000 7f0c68e75640 1 -- 172.21.15.110:0/345841976 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d6a51210 con 0x55f9d6ddeb20 2024-04-25T00:18:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.297+0000 7f0c5a7fc640 1 -- 172.21.15.110:0/2167773652 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0c5c01c070 con 0x7f0c64077ce0 2024-04-25T00:18:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.297+0000 7f0c68e75640 1 --2- 172.21.15.110:0/345841976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6de0fb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0ce340 tx=0x7f0c5c0d4b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4f8f52835abf04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.304+0000 7f0c5a7fc640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c500a2d90 0x7f0c500a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.305+0000 7f0c5a7fc640 1 -- 172.21.15.110:0/2167773652 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0c5c0d08d0 con 0x7f0c64077ce0 2024-04-25T00:18:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.306+0000 7f0c69e77640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c500a2d90 0x7f0c500a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.313+0000 7f0c69e77640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c500a2d90 0x7f0c500a5250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f9d6dda210 tx=0x7f0c54008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.314+0000 7f0c4f7fe640 1 -- 172.21.15.110:0/345841976 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c099450 con 0x55f9d6ddeb20 2024-04-25T00:18:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.314+0000 7f0c4f7fe640 1 -- 172.21.15.110:0/345841976 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c5c09d1b0 con 0x55f9d6ddeb20 2024-04-25T00:18:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.314+0000 7f0c4f7fe640 1 -- 172.21.15.110:0/345841976 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c09cb10 con 0x55f9d6ddeb20 2024-04-25T00:18:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.318+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/345841976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 msgr2=0x55f9d6de0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.318+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/345841976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6de0fb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0ce340 tx=0x7f0c5c0d4b90 comp rx=0 tx=0).stop 2024-04-25T00:18:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.322+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/345841976 shutdown_connections 2024-04-25T00:18:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.322+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/345841976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6de0fb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.322+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/345841976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd99b0 0x55f9d6dde4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.322+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/345841976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6dd93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.322+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/345841976 >> 172.21.15.110:0/345841976 conn(0x55f9d6dd4630 msgr2=0x55f9d6dd4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.327+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/345841976 shutdown_connections 2024-04-25T00:18:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.329+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/345841976 wait complete. 2024-04-25T00:18:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.330+0000 7f0c6e5ee740 1 -- start start 2024-04-25T00:18:11.331 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.330+0000 7f0c6e5ee740 1 Processor -- start 2024-04-25T00:18:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.330+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f9d6de1ad0 con 0x55f9d6ddeb20 2024-04-25T00:18:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f9d6cf8b50 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f9d6de1c50 con 0x55f9d6dd99b0 2024-04-25T00:18:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c6e5ee740 10 client.0 _ll_register_callbacks cb 0x55f9d6d33ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c69676640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c69676640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.110:37268/0 (socket says 172.21.15.110:37268) 2024-04-25T00:18:11.334 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.331+0000 7f0c69676640 1 -- 172.21.15.110:0/4184246533 learned_addr learned my addr 172.21.15.110:0/4184246533 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.332+0000 7f0c69e77640 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.333+0000 7f0c69676640 1 -- 172.21.15.110:0/4184246533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd99b0 msgr2=0x55f9d6de39e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:18:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.333+0000 7f0c69676640 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de39e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.333+0000 7f0c69676640 1 -- 172.21.15.110:0/4184246533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 msgr2=0x55f9d6dea9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.333+0000 7f0c69676640 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.333+0000 7f0c69676640 1 -- 172.21.15.110:0/4184246533 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d6b1d620 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.334+0000 7f0c69676640 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de34a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009850 tx=0x7f0c60004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=387e25196173831e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.337+0000 7f0c4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-25T00:18:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.337+0000 7f0c4cff9640 1 -- 172.21.15.110:0/4184246533 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60035030 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.337+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/4184246533 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c5c09d1b0 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.337+0000 7f0c4cff9640 1 -- 172.21.15.110:0/4184246533 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0c60003a80 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.341+0000 7f0c4cff9640 1 -- 172.21.15.110:0/4184246533 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c60003c80 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.341+0000 7f0c6e5ee740 10 client.26093 fetch_fsmap learned FSMap version 32 2024-04-25T00:18:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.341+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/4184246533 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9d6deddc0 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.343+0000 7f0c4cff9640 1 -- 172.21.15.110:0/4184246533 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c5c09d1b0 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.344+0000 7f0c4cff9640 1 -- 172.21.15.110:0/4184246533 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0c600455f0 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.344+0000 7f0c6e5ee740 10 client.26093 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:18:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.344+0000 7f0c6e5ee740 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.344+0000 7f0c6e5ee740 20 client.26093 populate_metadata read hostname 'smithi110' 2024-04-25T00:18:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.344+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f9d6def8f0 0x55f9d6df1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.344+0000 7f0c6e5ee740 4 client.26093 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.344+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/4184246533 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9d6d4a1f0 con 0x55f9d6def8f0 2024-04-25T00:18:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.348+0000 7f0c68e75640 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f9d6def8f0 0x55f9d6df1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:10 smithi173 ceph-mon[32402]: pgmap v835: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 825 op/s 2024-04-25T00:18:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.360+0000 7f0c68e75640 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f9d6def8f0 0x55f9d6df1d90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87ed8006d32cac59 server_cookie=a027b82d9ebd371f in_seq=0 out_seq=0 2024-04-25T00:18:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.360+0000 7f0c4cff9640 10 client.26093 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:18:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.366+0000 7f0c4cff9640 1 -- 172.21.15.110:0/4184246533 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9d6d4a1f0 con 0x55f9d6def8f0 2024-04-25T00:18:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.366+0000 7f0c4cff9640 10 client.26093 handle_command_reply: tid=0 2024-04-25T00:18:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.367+0000 7f0c6e5ee740 1 client.26093 shutdown 2024-04-25T00:18:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.374+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/4184246533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 msgr2=0x55f9d6de34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.374+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de34a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009850 tx=0x7f0c60004590 comp rx=0 tx=0).stop 2024-04-25T00:18:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.375+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/4184246533 shutdown_connections 2024-04-25T00:18:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.375+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f9d6def8f0 0x55f9d6df1d90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.375+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.375+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.375+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/4184246533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de34a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.375+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/4184246533 >> 172.21.15.110:0/4184246533 conn(0x55f9d6dd4630 msgr2=0x55f9d6dd3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.375+0000 7f0c4cff9640 0 client.26093 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:18:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.385+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/4184246533 shutdown_connections 2024-04-25T00:18:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.386+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/4184246533 wait complete. 2024-04-25T00:18:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.390+0000 7f0c6e5ee740 20 client.26093 trim_cache size 0 max 16384 2024-04-25T00:18:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.428+0000 7f0c6e5ee740 1 Processor -- start 2024-04-25T00:18:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.428+0000 7f0c6e5ee740 1 -- start start 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6df7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd99b0 0x55f9d6df7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48010d50 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4800e620 con 0x55f9d6dd99b0 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4800c470 con 0x55f9d6ddeb20 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c69e77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c69e77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.110:41632/0 (socket says 172.21.15.110:41632) 2024-04-25T00:18:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.429+0000 7f0c69e77640 1 -- 172.21.15.110:0/3101180634 learned_addr learned my addr 172.21.15.110:0/3101180634 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.430+0000 7f0c69676640 1 --2- 172.21.15.110:0/3101180634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6df7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.430+0000 7f0c69e77640 1 -- 172.21.15.110:0/3101180634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd99b0 msgr2=0x55f9d6df7ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:18:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.430+0000 7f0c69e77640 1 --2- 172.21.15.110:0/3101180634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd99b0 0x55f9d6df7ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.430+0000 7f0c69e77640 1 -- 172.21.15.110:0/3101180634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd8fc0 msgr2=0x55f9d6df7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.430+0000 7f0c69e77640 1 --2- 172.21.15.110:0/3101180634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6df7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.430+0000 7f0c69e77640 1 -- 172.21.15.110:0/3101180634 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d6dce850 con 0x55f9d6ddeb20 2024-04-25T00:18:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.431+0000 7f0c69e77640 1 --2- 172.21.15.110:0/3101180634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400b370 tx=0x7f0c54012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb43ea0642667b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.434+0000 7f0c4effd640 1 -- 172.21.15.110:0/3101180634 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5401b030 con 0x55f9d6ddeb20 2024-04-25T00:18:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.434+0000 7f0c4effd640 1 -- 172.21.15.110:0/3101180634 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c54026070 con 0x55f9d6ddeb20 2024-04-25T00:18:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.434+0000 7f0c4effd640 1 -- 172.21.15.110:0/3101180634 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54015670 con 0x55f9d6ddeb20 2024-04-25T00:18:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.435+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/3101180634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 msgr2=0x55f9d6dea940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.435+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/3101180634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400b370 tx=0x7f0c54012000 comp rx=0 tx=0).stop 2024-04-25T00:18:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.439+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/3101180634 shutdown_connections 2024-04-25T00:18:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.439+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/3101180634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6dea940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.439+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/3101180634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd99b0 0x55f9d6df7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.439+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/3101180634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6df7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.439+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/3101180634 >> 172.21.15.110:0/3101180634 conn(0x55f9d6dd4630 msgr2=0x55f9d6dd73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.443+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/3101180634 shutdown_connections 2024-04-25T00:18:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.446+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/3101180634 wait complete. 2024-04-25T00:18:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 1 -- start start 2024-04-25T00:18:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 1 Processor -- start 2024-04-25T00:18:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6e53ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4800cae0 con 0x55f9d6dd99b0 2024-04-25T00:18:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4800df50 con 0x55f9d6dd8fc0 2024-04-25T00:18:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4800d1b0 con 0x55f9d6ddeb20 2024-04-25T00:18:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.447+0000 7f0c6e5ee740 10 client.0 _ll_register_callbacks cb 0x7f0c60005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.448+0000 7f0c68e75640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.448+0000 7f0c68e75640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59924/0 (socket says 172.21.15.110:59924) 2024-04-25T00:18:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.448+0000 7f0c68e75640 1 -- 172.21.15.110:0/1299089060 learned_addr learned my addr 172.21.15.110:0/1299089060 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.448+0000 7f0c68e75640 1 -- 172.21.15.110:0/1299089060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 msgr2=0x55f9d6e53ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.448+0000 7f0c68e75640 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6e53ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.448+0000 7f0c68e75640 1 -- 172.21.15.110:0/1299089060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 msgr2=0x55f9d6de7990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:18:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.448+0000 7f0c68e75640 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de7990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.449+0000 7f0c68e75640 1 -- 172.21.15.110:0/1299089060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d6e544b0 con 0x55f9d6dd99b0 2024-04-25T00:18:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.451+0000 7f0c68e75640 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de7ed0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c099b80 tx=0x7f0c5c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e4a4e3207b2c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.451+0000 7f0c4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-04-25T00:18:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.455+0000 7f0c4cff9640 1 -- 172.21.15.110:0/1299089060 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c0b6060 con 0x55f9d6dd99b0 2024-04-25T00:18:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.455+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/1299089060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c54026070 con 0x55f9d6dd99b0 2024-04-25T00:18:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.455+0000 7f0c4cff9640 1 -- 172.21.15.110:0/1299089060 <== mon.0 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f0c5c099820 con 0x55f9d6dd99b0 2024-04-25T00:18:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.456+0000 7f0c4cff9640 1 -- 172.21.15.110:0/1299089060 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c5c09ccf0 con 0x55f9d6dd99b0 2024-04-25T00:18:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.459+0000 7f0c6e5ee740 10 client.16515 fetch_fsmap learned FSMap version 32 2024-04-25T00:18:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.459+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/1299089060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9d6e557c0 con 0x55f9d6dd99b0 2024-04-25T00:18:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.461+0000 7f0c4cff9640 1 -- 172.21.15.110:0/1299089060 <== mon.0 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c54026070 con 0x55f9d6dd99b0 2024-04-25T00:18:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.461+0000 7f0c4cff9640 1 -- 172.21.15.110:0/1299089060 <== mon.0 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0c5c0bd040 con 0x55f9d6dd99b0 2024-04-25T00:18:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.461+0000 7f0c6e5ee740 10 client.16515 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T00:18:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.461+0000 7f0c6e5ee740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.461+0000 7f0c6e5ee740 20 client.16515 populate_metadata read hostname 'smithi110' 2024-04-25T00:18:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.461+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f9d6e56150 0x55f9d6e58570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.461+0000 7f0c6e5ee740 4 client.16515 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:18:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.461+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/1299089060 --> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9d6d44540 con 0x55f9d6e56150 2024-04-25T00:18:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.463+0000 7f0c69676640 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f9d6e56150 0x55f9d6e58570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.466+0000 7f0c69676640 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f9d6e56150 0x55f9d6e58570 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec03df2434560cb server_cookie=e5d27f22a5eb2559 in_seq=0 out_seq=0 2024-04-25T00:18:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:11.467+0000 7f0c4cff9640 10 client.16515 ms_handle_connect on v2:172.21.15.173:6838/2519479764 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.208+0000 7f0c4cff9640 1 -- 172.21.15.110:0/1299089060 <== mds.0 v2:172.21.15.173:6838/2519479764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9d6d44540 con 0x55f9d6e56150 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.208+0000 7f0c4cff9640 10 client.16515 handle_command_reply: tid=0 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.208+0000 7f0c6e5ee740 1 client.16515 shutdown 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.208+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/1299089060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd99b0 msgr2=0x55f9d6de7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.208+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de7ed0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c099b80 tx=0x7f0c5c00bf90 comp rx=0 tx=0).stop 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/1299089060 shutdown_connections 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.173:6838/2519479764,v1:172.21.15.173:6839/2519479764] conn(0x55f9d6e56150 0x55f9d6e58570 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f9d6ddeb20 0x55f9d6e53ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9d6dd99b0 0x55f9d6de7ed0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 1 --2- 172.21.15.110:0/1299089060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9d6dd8fc0 0x55f9d6de7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/1299089060 >> 172.21.15.110:0/1299089060 conn(0x55f9d6dd4630 msgr2=0x55f9d6dd7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c4cff9640 0 client.16515 ms_handle_reset on v2:172.21.15.173:6838/2519479764 2024-04-25T00:18:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/1299089060 shutdown_connections 2024-04-25T00:18:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 1 -- 172.21.15.110:0/1299089060 wait complete. 2024-04-25T00:18:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.209+0000 7f0c6e5ee740 20 client.16515 trim_cache size 0 max 16384 2024-04-25T00:18:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:{ 2024-04-25T00:18:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "message": "", 2024-04-25T00:18:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout: "return_code": 0 2024-04-25T00:18:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stdout:} 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 -- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c500a2d90 msgr2=0x7f0c500a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c500a2d90 0x7f0c500a5250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f9d6dda210 tx=0x7f0c54008040 comp rx=0 tx=0).stop 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 -- 172.21.15.110:0/2167773652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64077ce0 msgr2=0x7f0c6419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64077ce0 0x7f0c6419e770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c00bd50 tx=0x7f0c5c00be40 comp rx=0 tx=0).stop 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 -- 172.21.15.110:0/2167773652 shutdown_connections 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f0c500a2d90 0x7f0c500a5250 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c641a37d0 0x7f0c641a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64077ce0 0x7f0c6419e770 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 --2- 172.21.15.110:0/2167773652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c64077340 0x7f0c6419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 -- 172.21.15.110:0/2167773652 >> 172.21.15.110:0/2167773652 conn(0x7f0c640fed70 msgr2=0x7f0c641021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 -- 172.21.15.110:0/2167773652 shutdown_connections 2024-04-25T00:18:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi110.front.sepia.ceph.com.smithi110.stderr:2024-04-25T00:18:12.210+0000 7f0c6a678640 1 -- 172.21.15.110:0/2167773652 wait complete. 2024-04-25T00:18:13.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:12 smithi188 ceph-mon[33285]: pgmap v836: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-04-25T00:18:13.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:12 smithi110 ceph-mon[29187]: pgmap v836: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-04-25T00:18:13.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:12 smithi173 ceph-mon[32402]: pgmap v836: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-04-25T00:18:14.059 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-04-25T00:18:14.373 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/: 2024-04-25T00:18:14.373 INFO:teuthology.orchestra.run.smithi110.stdout:total 2 2024-04-25T00:18:14.373 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:02 . 2024-04-25T00:18:14.373 INFO:teuthology.orchestra.run.smithi110.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:02 .. 2024-04-25T00:18:14.373 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 data 2024-04-25T00:18:14.373 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data: 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:total 46 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 . 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:02 .. 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577582645 Apr 25 00:03 base 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Apr 25 00:02 current_logfiles 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 Apr 25 00:02 global 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 716 Apr 25 00:02 log 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_commit_ts 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_dynshmem 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Apr 25 00:02 pg_hba.conf 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Apr 25 00:02 pg_ident.conf 2024-04-25T00:18:14.374 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:18 pg_logical 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 pg_multixact 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_notify 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_replslot 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_serial 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_snapshots 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Apr 25 00:18 pg_stat 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:18 pg_stat_tmp 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:18 pg_subtrans 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_tblspc 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 pg_twophase 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 25 00:02 PG_VERSION 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 905969664 Apr 25 00:18 pg_wal 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 204800 Apr 25 00:02 pg_xact 2024-04-25T00:18:14.375 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Apr 25 00:02 postgresql.auto.conf 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Apr 25 00:02 postgresql.conf 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Apr 25 00:02 postmaster.opts 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base: 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:total 3 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577582645 Apr 25 00:03 . 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 1 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 13493 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 561656367 Apr 25 00:08 13494 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:03 pgsql_tmp 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base/1: 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:total 7778 2024-04-25T00:18:14.376 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 . 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577582645 Apr 25 00:03 .. 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 112 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 113 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 1247 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1247_fsm 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1247_vm 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 25 00:02 1249 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1249_fsm 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1249_vm 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 25 00:02 1255 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1255_fsm 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1255_vm 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 25 00:02 1259 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1259_fsm 2024-04-25T00:18:14.377 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1259_vm 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 25 00:02 13344 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13344_fsm 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13344_vm 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13346 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13348 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13349_fsm 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349_vm 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13351 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13353 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13354_fsm 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354_vm 2024-04-25T00:18:14.378 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13356 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13358 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13359_fsm 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359_vm 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13361 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13363 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1417 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1418 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 174 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 175 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2187 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2224 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2328 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2336 2024-04-25T00:18:14.379 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2337 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2579 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2600 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2600_fsm 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2600_vm 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2601_fsm 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601_vm 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2602 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2602_fsm 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2602_vm 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2603 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2603_fsm 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2603_vm 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2604 2024-04-25T00:18:14.380 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2605 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2605_fsm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2605_vm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2606_fsm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606_vm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2607 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2607_fsm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2607_vm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 25 00:02 2608 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2608_fsm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2608_vm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 25 00:02 2609 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2609_fsm 2024-04-25T00:18:14.381 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2609_vm 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2610 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2610_fsm 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2610_vm 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2611 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2612_fsm 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612_vm 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2613 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2615_fsm 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615_vm 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2616 2024-04-25T00:18:14.382 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2616_fsm 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2616_vm 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 25 00:02 2617 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2617_fsm 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2617_vm 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 25 00:02 2618 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2618_fsm 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2618_vm 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 25 00:02 2619 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2619_fsm 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2619_vm 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2620 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2650 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2651 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2652 2024-04-25T00:18:14.383 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2653 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2654 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2655 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2656 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2657 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 25 00:02 2658 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 2659 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2660 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2661 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2662 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2663 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2664 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2665 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2666 2024-04-25T00:18:14.384 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2667 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2668 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2669 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2670 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 25 00:02 2673 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 25 00:02 2674 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 25 00:02 2675 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2678 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2679 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2680 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2681 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2682 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2683 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2684 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2685 2024-04-25T00:18:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2686 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2687 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2688 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2689 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 25 00:02 2690 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 25 00:02 2691 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2692 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2693 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2696 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2699 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2701 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2702 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2703 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2704 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2753 2024-04-25T00:18:14.386 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2753_fsm 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2753_vm 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2754 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2755 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2756 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2757 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2830 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2831 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2832 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2833 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2834 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2835 2024-04-25T00:18:14.387 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2836 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2837 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 25 00:02 2838 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2838_fsm 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2838_vm 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2839 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840_fsm 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2840_vm 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2841 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2995 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2996 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079 2024-04-25T00:18:14.388 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3079_fsm 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079_vm 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3080 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3081 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3085 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3118 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3119 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 25 00:02 3164 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3256 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3257 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3258 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3350 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3351 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3379 2024-04-25T00:18:14.389 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3380 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3381 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3394 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3394_fsm 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3394_vm 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3395 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3429 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3430 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3431 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3433 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3439 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3440 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3455 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 25 00:02 3456 2024-04-25T00:18:14.390 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3456_fsm 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3456_vm 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3466 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3467 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3468 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3501 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3502 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3503 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3534 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3541_fsm 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541_vm 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3542 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3574 2024-04-25T00:18:14.391 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3575 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3576 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3596 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3597 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3598 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3599 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3600_fsm 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600_vm 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3601_fsm 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601_vm 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3602_fsm 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602_vm 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603 2024-04-25T00:18:14.392 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603_fsm 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3603_vm 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3604 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3605 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3606 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3607 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3608 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 3609 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3712 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3764_fsm 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764_vm 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3766 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3767 2024-04-25T00:18:14.393 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3997 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4143 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4144 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4145 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4146 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4147 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4148 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4149 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4150 2024-04-25T00:18:14.394 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4151 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4152 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4153 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4154 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4155 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4156 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4157 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4158 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4159 2024-04-25T00:18:14.612 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4160 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4161 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4162 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4163 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4164 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4165 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4166 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4167 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4168 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4169 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4170 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4171 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4172 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4173 2024-04-25T00:18:14.613 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4174 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 5002 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 548 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 549 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6102 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6104 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6106 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6110 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6111 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6112 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6113 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6117 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 826 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 827 2024-04-25T00:18:14.614 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 828 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 25 00:02 pg_filenode.map 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 25 00:02 PG_VERSION 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:total 7778 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 25 00:02 . 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577582645 Apr 25 00:03 .. 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 112 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 113 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 1247 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1247_fsm 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1247_vm 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 25 00:02 1249 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1249_fsm 2024-04-25T00:18:14.615 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1249_vm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 25 00:02 1255 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1255_fsm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1255_vm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 25 00:02 1259 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1259_fsm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1259_vm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 25 00:02 13344 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13344_fsm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13344_vm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13346 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13348 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13349_fsm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349_vm 2024-04-25T00:18:14.616 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13351 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13353 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13354_fsm 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354_vm 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13356 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13358 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13359_fsm 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359_vm 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13361 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13363 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1417 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1418 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 174 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 175 2024-04-25T00:18:14.617 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2187 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2224 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2328 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2336 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2337 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2579 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2600 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2600_fsm 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2600_vm 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2601_fsm 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601_vm 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2602 2024-04-25T00:18:14.618 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2602_fsm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2602_vm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2603 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2603_fsm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2603_vm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2604 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2605 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2605_fsm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2605_vm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2606_fsm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2606_vm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2607 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2607_fsm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2607_vm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 25 00:02 2608 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2608_fsm 2024-04-25T00:18:14.619 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2608_vm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 25 00:02 2609 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2609_fsm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2609_vm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2610 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2610_fsm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2610_vm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2611 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2612_fsm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612_vm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2613 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2615_fsm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615_vm 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2616 2024-04-25T00:18:14.620 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2616_fsm 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2616_vm 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 25 00:02 2617 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2617_fsm 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2617_vm 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 25 00:02 2618 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2618_fsm 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2618_vm 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 25 00:02 2619 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2619_fsm 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2619_vm 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2620 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2650 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2651 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2652 2024-04-25T00:18:14.621 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2653 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2654 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2655 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2656 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2657 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 25 00:02 2658 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:02 2659 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2660 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2661 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2662 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2663 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2664 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2665 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2666 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2667 2024-04-25T00:18:14.622 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2668 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2669 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2670 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 25 00:02 2673 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 25 00:02 2674 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 25 00:02 2675 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2678 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2679 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2680 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2681 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2682 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2683 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2684 2024-04-25T00:18:14.623 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2685 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2686 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2687 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2688 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2689 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 25 00:02 2690 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 25 00:02 2691 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2692 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2693 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2696 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2699 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2701 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2702 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2703 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2704 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2753 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2753_fsm 2024-04-25T00:18:14.624 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2753_vm 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2754 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2755 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2756 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2757 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2830 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2831 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2832 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2833 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2834 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2835 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2836 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2837 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 25 00:02 2838 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2838_fsm 2024-04-25T00:18:14.625 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2838_vm 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2839 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840_fsm 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2840_vm 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2841 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2995 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2996 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3079_fsm 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079_vm 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3080 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3081 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3085 2024-04-25T00:18:14.626 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3118 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3119 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 25 00:02 3164 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3256 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3257 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3258 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3350 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3351 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3379 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3380 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3381 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3394 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3394_fsm 2024-04-25T00:18:14.627 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3394_vm 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3395 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3429 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3430 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3431 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3433 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3439 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3440 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3455 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 25 00:02 3456 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3456_fsm 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3456_vm 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3466 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3467 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3468 2024-04-25T00:18:14.628 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3501 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3502 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3503 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3534 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3541_fsm 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541_vm 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3542 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3574 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3575 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3576 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3596 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3597 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3598 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3599 2024-04-25T00:18:14.629 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3600_fsm 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600_vm 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3601_fsm 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601_vm 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3602_fsm 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602_vm 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603_fsm 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3603_vm 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3604 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3605 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3606 2024-04-25T00:18:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3607 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3608 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 3609 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3712 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3764_fsm 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764_vm 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3766 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3767 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3997 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4143 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4144 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4145 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4146 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4147 2024-04-25T00:18:14.631 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4148 2024-04-25T00:18:14.632 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4149 2024-04-25T00:18:14.632 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4150 2024-04-25T00:18:14.632 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4151 2024-04-25T00:18:14.632 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4152 2024-04-25T00:18:14.632 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4153 2024-04-25T00:18:14.879 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4154 2024-04-25T00:18:14.879 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4155 2024-04-25T00:18:14.879 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4156 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4157 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4158 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4159 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4160 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4161 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4162 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4163 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4164 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4165 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4166 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4167 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4168 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4169 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4170 2024-04-25T00:18:14.880 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4171 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4172 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4173 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4174 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 5002 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 548 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 549 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6102 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6104 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6106 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6110 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6111 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6112 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6113 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6117 2024-04-25T00:18:14.881 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 826 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 827 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 828 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 25 00:02 pg_filenode.map 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 25 00:02 PG_VERSION 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:total 548495 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 561656367 Apr 25 00:08 . 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577582645 Apr 25 00:03 .. 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 112 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 113 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:03 1247 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1247_fsm 2024-04-25T00:18:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:03 1247_vm 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 25 00:03 1249 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:03 1249_fsm 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:03 1249_vm 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 25 00:02 1255 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1255_fsm 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1255_vm 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 25 00:17 1259 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1259_fsm 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:03 1259_vm 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 25 00:02 13344 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13344_fsm 2024-04-25T00:18:14.883 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13344_vm 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13346 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13348 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13349_fsm 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13349_vm 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13351 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13353 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13354_fsm 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13354_vm 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13356 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13358 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 13359_fsm 2024-04-25T00:18:14.884 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13359_vm 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 13361 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 13363 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1417 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 1418 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 437911552 Apr 25 00:18 16396 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Apr 25 00:18 16396_fsm 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:14 16396_vm 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 638976 Apr 25 00:18 16397 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:17 16397_fsm 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:17 16397_vm 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1048576 Apr 25 00:18 16399 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:17 16399_fsm 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:17 16399_vm 2024-04-25T00:18:14.885 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:18 16400 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:18 16402 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Apr 25 00:18 16404 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 41689088 Apr 25 00:18 16406 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:18 16406_fsm 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:14 16406_vm 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 174 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 175 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2187 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2224 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2328 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2336 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2337 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:03 2579 2024-04-25T00:18:14.886 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2600 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2600_fsm 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2600_vm 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2601_fsm 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2601_vm 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2602 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2602_fsm 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2602_vm 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2603 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2603_fsm 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2603_vm 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2604 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2605 2024-04-25T00:18:14.887 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2605_fsm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2605_vm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:03 2606 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2606_fsm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:03 2606_vm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2607 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2607_fsm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2607_vm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 25 00:03 2608 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2608_fsm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:03 2608_vm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 25 00:02 2609 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2609_fsm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2609_vm 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:03 2610 2024-04-25T00:18:14.888 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2610_fsm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:03 2610_vm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2611 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2612_fsm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2612_vm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2613 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2615_fsm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2615_vm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2616 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2616_fsm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2616_vm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 25 00:02 2617 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2617_fsm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2617_vm 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 25 00:02 2618 2024-04-25T00:18:14.889 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2618_fsm 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2618_vm 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Apr 25 00:17 2619 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:07 2619_fsm 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:03 2619_vm 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2620 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2650 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2651 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2652 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2653 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2654 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2655 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2656 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2657 2024-04-25T00:18:14.890 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 25 00:03 2658 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 25 00:03 2659 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2660 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2661 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:03 2662 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:03 2663 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:03 2664 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:03 2665 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:03 2666 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:03 2667 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2668 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2669 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2670 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 25 00:03 2673 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 25 00:03 2674 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 25 00:02 2675 2024-04-25T00:18:14.891 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:03 2678 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:03 2679 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2680 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2681 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2682 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2683 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2684 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2685 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2686 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2687 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 2688 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 25 00:02 2689 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 25 00:02 2690 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 25 00:02 2691 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2692 2024-04-25T00:18:14.892 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2693 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:08 2696 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2699 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2701 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2702 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:03 2703 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:03 2704 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2753 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2753_fsm 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2753_vm 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2754 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2755 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2756 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 2757 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2830 2024-04-25T00:18:14.893 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2831 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2832 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2833 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2834 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2835 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2836 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2837 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 25 00:02 2838 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2838_fsm 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2838_vm 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2839 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2840_fsm 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2840_vm 2024-04-25T00:18:14.894 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2841 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2995 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2996 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3079_fsm 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3079_vm 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3080 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3081 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:02 3085 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3118 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3119 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 25 00:02 3164 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3256 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3257 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3258 2024-04-25T00:18:14.895 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3350 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3351 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3379 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3380 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3381 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3394 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3394_fsm 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3394_vm 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3395 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3429 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3430 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3431 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3433 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3439 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3440 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:03 3455 2024-04-25T00:18:14.896 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 25 00:02 3456 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3456_fsm 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3456_vm 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3466 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3467 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3468 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3501 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3502 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3503 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3534 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3541_fsm 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3541_vm 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3542 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3574 2024-04-25T00:18:14.897 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3575 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3576 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3596 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3597 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3598 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3599 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3600_fsm 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3600_vm 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3601_fsm 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3601_vm 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602 2024-04-25T00:18:14.898 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3602_fsm 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3602_vm 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3603_fsm 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3603_vm 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3604 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3605 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3606 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3607 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3608 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 25 00:02 3609 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3712 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 3764_fsm 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3764_vm 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3766 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 3767 2024-04-25T00:18:14.899 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3997 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4143 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4144 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4145 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4146 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4147 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4148 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4149 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4150 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4151 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4152 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4153 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4154 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4155 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4156 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4157 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4158 2024-04-25T00:18:14.900 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4159 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4160 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4161 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4162 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4163 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4164 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4165 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4166 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4167 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4168 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4169 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4170 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4171 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4172 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4173 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4174 2024-04-25T00:18:14.901 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 5002 2024-04-25T00:18:14.902 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 548 2024-04-25T00:18:14.902 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 549 2024-04-25T00:18:14.902 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6102 2024-04-25T00:18:14.902 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6104 2024-04-25T00:18:14.902 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6106 2024-04-25T00:18:14.902 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6110 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6111 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6112 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6113 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6117 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 826 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 827 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 828 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 25 00:02 pg_filenode.map 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Apr 25 00:02 pg_internal.init 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 25 00:02 PG_VERSION 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.925 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:03 . 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577582645 Apr 25 00:03 .. 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/global: 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:total 561 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 Apr 25 00:02 . 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1213 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1213_fsm 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1213_vm 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1214 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1214_fsm 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1214_vm 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 1232 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 1233 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1260 2024-04-25T00:18:14.926 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1260_fsm 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1260_vm 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1261 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1261_fsm 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1261_vm 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1262 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 1262_fsm 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 1262_vm 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2396 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 25 00:02 2396_fsm 2024-04-25T00:18:14.927 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2396_vm 2024-04-25T00:18:14.928 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2397 2024-04-25T00:18:14.928 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2671 2024-04-25T00:18:14.928 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2672 2024-04-25T00:18:14.928 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2676 2024-04-25T00:18:14.928 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2677 2024-04-25T00:18:14.928 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2694 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2695 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2697 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 2698 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2846 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2847 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2964 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2965 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 2966 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 2967 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 3592 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 3593 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4060 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4061 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4175 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4176 2024-04-25T00:18:14.940 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4177 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4178 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4181 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4182 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4183 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4184 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 4185 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 4186 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6000 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6001 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6002 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 6100 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6114 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 6115 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:18 pg_control 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 25 00:02 pg_filenode.map 2024-04-25T00:18:14.941 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Apr 25 00:02 pg_internal.init 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/log: 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:total 3 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 716 Apr 25 00:02 . 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 Apr 25 00:18 postgresql-Thu.log 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-04-25T00:18:14.942 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.943 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:14.943 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.943 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.943 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-04-25T00:18:14.943 INFO:teuthology.orchestra.run.smithi110.stdout:total 3 2024-04-25T00:18:14.980 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:18 . 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 mappings 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:18 replorigin_checkpoint 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 snapshots 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:18 .. 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.981 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 25 00:18 .. 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:total 2 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 . 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 members 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 offsets 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-04-25T00:18:14.982 INFO:teuthology.orchestra.run.smithi110.stdout:total 9 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 . 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 .. 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 0000 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:total 9 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:02 . 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 25 00:02 .. 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 25 00:07 0000 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:14.984 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:total 16 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Apr 25 00:18 . 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Apr 25 00:18 db_0.stat 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Apr 25 00:18 db_13494.stat 2024-04-25T00:18:14.985 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Apr 25 00:18 global.stat 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:18 . 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout:total 41 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:18 . 2024-04-25T00:18:14.986 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:15.022 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 25 00:18 000C 2024-04-25T00:18:15.022 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:total 868354 2024-04-25T00:18:15.023 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 905969664 Apr 25 00:18 . 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:18 00000001000000000000006F 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:08 000000010000000000000070 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:09 000000010000000000000071 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:07 000000010000000000000072 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:10 000000010000000000000073 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:09 000000010000000000000074 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:08 000000010000000000000075 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:08 000000010000000000000076 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 000000010000000000000077 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 000000010000000000000078 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 000000010000000000000079 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 00000001000000000000007A 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:14 00000001000000000000007B 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:14 00000001000000000000007C 2024-04-25T00:18:15.024 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 00000001000000000000007D 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 00000001000000000000007E 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 00000001000000000000007F 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 000000010000000000000080 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:15 000000010000000000000081 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 000000010000000000000082 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 000000010000000000000083 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:16 000000010000000000000084 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 000000010000000000000085 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:14 000000010000000000000086 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 000000010000000000000087 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:14 000000010000000000000088 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 000000010000000000000089 2024-04-25T00:18:15.025 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:14 00000001000000000000008A 2024-04-25T00:18:15.028 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:15 00000001000000000000008B 2024-04-25T00:18:15.028 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 00000001000000000000008C 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 00000001000000000000008D 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 00000001000000000000008E 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 00000001000000000000008F 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 000000010000000000000090 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:16 000000010000000000000091 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:13 000000010000000000000092 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 000000010000000000000093 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 000000010000000000000094 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:12 000000010000000000000095 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:15 000000010000000000000096 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 000000010000000000000097 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 000000010000000000000098 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 000000010000000000000099 2024-04-25T00:18:15.029 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 00000001000000000000009A 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 00000001000000000000009B 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 00000001000000000000009C 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 00000001000000000000009D 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 00000001000000000000009E 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 00000001000000000000009F 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 0000000100000000000000A0 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 0000000100000000000000A1 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:18 0000000100000000000000A2 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 25 00:17 0000000100000000000000A3 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 archive_status 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-04-25T00:18:15.030 INFO:teuthology.orchestra.run.smithi110.stdout:total 1 2024-04-25T00:18:15.031 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 25 00:02 . 2024-04-25T00:18:15.031 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 905969664 Apr 25 00:18 .. 2024-04-25T00:18:15.031 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:15.031 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-04-25T00:18:15.031 INFO:teuthology.orchestra.run.smithi110.stdout:total 201 2024-04-25T00:18:15.031 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 204800 Apr 25 00:02 . 2024-04-25T00:18:15.031 INFO:teuthology.orchestra.run.smithi110.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1484427993 Apr 25 00:18 .. 2024-04-25T00:18:15.031 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 204800 Apr 25 00:18 0000 2024-04-25T00:18:15.032 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T00:18:15.043 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T00:18:15.051 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T00:18:15.062 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:15.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:14 smithi188 ceph-mon[33285]: pgmap v837: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-04-25T00:18:15.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:14 smithi110 ceph-mon[29187]: pgmap v837: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-04-25T00:18:15.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 -- 172.21.15.110:0/2389482634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610102230 msgr2=0x7fd610108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 --2- 172.21.15.110:0/2389482634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610102230 0x7fd610108ac0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd608009ca0 tx=0x7fd60802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 -- 172.21.15.110:0/2389482634 shutdown_connections 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 --2- 172.21.15.110:0/2389482634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610102230 0x7fd610108ac0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 --2- 172.21.15.110:0/2389482634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd610101870 0x7fd610101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 --2- 172.21.15.110:0/2389482634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd610100680 0x7fd610100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 -- 172.21.15.110:0/2389482634 >> 172.21.15.110:0/2389482634 conn(0x7fd6100fbbf0 msgr2=0x7fd6100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 -- 172.21.15.110:0/2389482634 shutdown_connections 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.295+0000 7fd6166c4640 1 -- 172.21.15.110:0/2389482634 wait complete. 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd6166c4640 1 Processor -- start 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd6166c4640 1 -- start start 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd6166c4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610100680 0x7fd610069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd6166c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd610101870 0x7fd61019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd6166c4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd610102230 0x7fd61019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd6166c4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd61010c1b0 con 0x7fd610100680 2024-04-25T00:18:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd6166c4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd61010c030 con 0x7fd610102230 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd6166c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd61010c330 con 0x7fd610101870 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610100680 0x7fd610069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610100680 0x7fd610069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59948/0 (socket says 172.21.15.110:59948) 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60ffff640 1 -- 172.21.15.110:0/544965646 learned_addr learned my addr 172.21.15.110:0/544965646 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60f7fe640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd610101870 0x7fd61019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60ffff640 1 -- 172.21.15.110:0/544965646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd610101870 msgr2=0x7fd61019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60ffff640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd610101870 0x7fd61019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd614c61640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd610102230 0x7fd61019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60ffff640 1 -- 172.21.15.110:0/544965646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd610102230 msgr2=0x7fd61019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60ffff640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd610102230 0x7fd61019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd60ffff640 1 -- 172.21.15.110:0/544965646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61019f520 con 0x7fd610100680 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.296+0000 7fd614c61640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd610102230 0x7fd61019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.297+0000 7fd60ffff640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610100680 0x7fd610069460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd60000d950 tx=0x7fd60000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.297+0000 7fd60d7fa640 1 -- 172.21.15.110:0/544965646 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600017070 con 0x7fd610100680 2024-04-25T00:18:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.297+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61019f810 con 0x7fd610100680 2024-04-25T00:18:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.297+0000 7fd60d7fa640 1 -- 172.21.15.110:0/544965646 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd600009c30 con 0x7fd610100680 2024-04-25T00:18:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.297+0000 7fd60d7fa640 1 -- 172.21.15.110:0/544965646 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60000f620 con 0x7fd610100680 2024-04-25T00:18:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.297+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61019fd60 con 0x7fd610100680 2024-04-25T00:18:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.298+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6101079a0 con 0x7fd610100680 2024-04-25T00:18:15.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.299+0000 7fd60d7fa640 1 -- 172.21.15.110:0/544965646 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd60001a070 con 0x7fd610100680 2024-04-25T00:18:15.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.299+0000 7fd60d7fa640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5f80a2b90 0x7fd5f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.299+0000 7fd60f7fe640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5f80a2b90 0x7fd5f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.299+0000 7fd60d7fa640 1 -- 172.21.15.110:0/544965646 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fd6000ceb50 con 0x7fd610100680 2024-04-25T00:18:15.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.300+0000 7fd60f7fe640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5f80a2b90 0x7fd5f80a5050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd604005e10 tx=0x7fd604005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:15.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.304+0000 7fd60d7fa640 1 -- 172.21.15.110:0/544965646 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd600009de0 con 0x7fd610100680 2024-04-25T00:18:15.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:14 smithi173 ceph-mon[32402]: pgmap v837: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-04-25T00:18:15.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.481+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd61002d2f0 con 0x7fd610100680 2024-04-25T00:18:15.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.483+0000 7fd60d7fa640 1 -- 172.21.15.110:0/544965646 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7fd60009bae0 con 0x7fd610100680 2024-04-25T00:18:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:15.484 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:15.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5f80a2b90 msgr2=0x7fd5f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5f80a2b90 0x7fd5f80a5050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd604005e10 tx=0x7fd604005da0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610100680 msgr2=0x7fd610069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610100680 0x7fd610069460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd60000d950 tx=0x7fd60000de10 comp rx=0 tx=0).stop 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 shutdown_connections 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd5f80a2b90 0x7fd5f80a5050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd610102230 0x7fd61019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd610101870 0x7fd61019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 --2- 172.21.15.110:0/544965646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd610100680 0x7fd610069460 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 >> 172.21.15.110:0/544965646 conn(0x7fd6100fbbf0 msgr2=0x7fd610104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 shutdown_connections 2024-04-25T00:18:15.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.485+0000 7fd6166c4640 1 -- 172.21.15.110:0/544965646 wait complete. 2024-04-25T00:18:15.496 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:01.224042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:02.090769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:15.497 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:18:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- 172.21.15.110:0/3873012753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 msgr2=0x7f3784108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 --2- 172.21.15.110:0/3873012753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 0x7f3784108ad0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f377c009ca0 tx=0x7f377c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- 172.21.15.110:0/3873012753 shutdown_connections 2024-04-25T00:18:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 --2- 172.21.15.110:0/3873012753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 0x7f3784108ad0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 --2- 172.21.15.110:0/3873012753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3784101880 0x7f3784101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 --2- 172.21.15.110:0/3873012753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100690 0x7f3784100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- 172.21.15.110:0/3873012753 >> 172.21.15.110:0/3873012753 conn(0x7f37840fbc00 msgr2=0x7f37840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- 172.21.15.110:0/3873012753 shutdown_connections 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- 172.21.15.110:0/3873012753 wait complete. 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 Processor -- start 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- start start 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100690 0x7f3784197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3784101880 0x7f3784197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 0x7f378419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f378410d2e0 con 0x7f3784102240 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f378410d160 con 0x7f3784100690 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.598+0000 7f378c2ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f378410d460 con 0x7f3784101880 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 0x7f378419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 0x7f378419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59970/0 (socket says 172.21.15.110:59970) 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 -- 172.21.15.110:0/2257424225 learned_addr learned my addr 172.21.15.110:0/2257424225 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a08a640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100690 0x7f3784197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f3789889640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3784101880 0x7f3784197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 -- 172.21.15.110:0/2257424225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3784101880 msgr2=0x7f3784197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3784101880 0x7f3784197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 -- 172.21.15.110:0/2257424225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100690 msgr2=0x7f3784197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100690 0x7f3784197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 -- 172.21.15.110:0/2257424225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378419f590 con 0x7f3784102240 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378a88b640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 0x7f378419ef20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f377c009dc0 tx=0x7f377c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f37777fe640 1 -- 172.21.15.110:0/2257424225 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c032ec0 con 0x7f3784102240 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f37777fe640 1 -- 172.21.15.110:0/2257424225 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f377c0044f0 con 0x7f3784102240 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f37777fe640 1 -- 172.21.15.110:0/2257424225 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c030d40 con 0x7f3784102240 2024-04-25T00:18:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378419f820 con 0x7f3784102240 2024-04-25T00:18:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.599+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378419fd70 con 0x7f3784102240 2024-04-25T00:18:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.600+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37841079b0 con 0x7f3784102240 2024-04-25T00:18:15.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.601+0000 7f37777fe640 1 -- 172.21.15.110:0/2257424225 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f377c004040 con 0x7f3784102240 2024-04-25T00:18:15.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.602+0000 7f37777fe640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37700a2db0 0x7f37700a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.602+0000 7f378a08a640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37700a2db0 0x7f37700a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.602+0000 7f37777fe640 1 -- 172.21.15.110:0/2257424225 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f377c0f0650 con 0x7f3784102240 2024-04-25T00:18:15.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.602+0000 7f378a08a640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37700a2db0 0x7f37700a5270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f37841016e0 tx=0x7f3778005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:15.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.606+0000 7f37777fe640 1 -- 172.21.15.110:0/2257424225 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f377c0bd5e0 con 0x7f3784102240 2024-04-25T00:18:15.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.787+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f378402d390 con 0x7f3784102240 2024-04-25T00:18:15.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.787+0000 7f37777fe640 1 -- 172.21.15.110:0/2257424225 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f377c0bcee0 con 0x7f3784102240 2024-04-25T00:18:15.788 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37700a2db0 msgr2=0x7f37700a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37700a2db0 0x7f37700a5270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f37841016e0 tx=0x7f3778005c60 comp rx=0 tx=0).stop 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 msgr2=0x7f378419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 0x7f378419ef20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f377c009dc0 tx=0x7f377c003ba0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 shutdown_connections 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f37700a2db0 0x7f37700a5270 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3784102240 0x7f378419ef20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3784101880 0x7f3784197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 --2- 172.21.15.110:0/2257424225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3784100690 0x7f3784197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 >> 172.21.15.110:0/2257424225 conn(0x7f37840fbc00 msgr2=0x7f3784104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:15.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 shutdown_connections 2024-04-25T00:18:15.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.789+0000 7f378c2ee640 1 -- 172.21.15.110:0/2257424225 wait complete. 2024-04-25T00:18:15.800 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:18:15.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 -- 172.21.15.110:0/2646224532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0101790 msgr2=0x7f37b0101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 --2- 172.21.15.110:0/2646224532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0101790 0x7f37b0101bb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009ca0 tx=0x7f37ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 -- 172.21.15.110:0/2646224532 shutdown_connections 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 --2- 172.21.15.110:0/2646224532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37b0103340 0x7f37b0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 --2- 172.21.15.110:0/2646224532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37b0102980 0x7f37b0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 --2- 172.21.15.110:0/2646224532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0101790 0x7f37b0101bb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 -- 172.21.15.110:0/2646224532 >> 172.21.15.110:0/2646224532 conn(0x7f37b00fcd20 msgr2=0x7f37b00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 -- 172.21.15.110:0/2646224532 shutdown_connections 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.902+0000 7f37b845d640 1 -- 172.21.15.110:0/2646224532 wait complete. 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b845d640 1 Processor -- start 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b845d640 1 -- start start 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b845d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37b0101790 0x7f37b01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b845d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0102980 0x7f37b0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b845d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37b0103340 0x7f37b019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b845d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37b010d290 con 0x7f37b0102980 2024-04-25T00:18:15.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b845d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37b010d110 con 0x7f37b0103340 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b845d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f37b010d410 con 0x7f37b0101790 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b59f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0102980 0x7f37b0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b59f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0102980 0x7f37b0197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59996/0 (socket says 172.21.15.110:59996) 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b59f8640 1 -- 172.21.15.110:0/1269271007 learned_addr learned my addr 172.21.15.110:0/1269271007 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b69fa640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37b0103340 0x7f37b019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b61f9640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37b0101790 0x7f37b01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b59f8640 1 -- 172.21.15.110:0/1269271007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37b0101790 msgr2=0x7f37b01979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b59f8640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37b0101790 0x7f37b01979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b59f8640 1 -- 172.21.15.110:0/1269271007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37b0103340 msgr2=0x7f37b019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b59f8640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37b0103340 0x7f37b019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b59f8640 1 -- 172.21.15.110:0/1269271007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b019f610 con 0x7f37b0102980 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.903+0000 7f37b69fa640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37b0103340 0x7f37b019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.904+0000 7f37b59f8640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0102980 0x7f37b0197f30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f37a000d950 tx=0x7f37a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:15.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.904+0000 7f37a77fe640 1 -- 172.21.15.110:0/1269271007 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0017070 con 0x7f37b0102980 2024-04-25T00:18:15.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.904+0000 7f37a77fe640 1 -- 172.21.15.110:0/1269271007 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37a0009c30 con 0x7f37b0102980 2024-04-25T00:18:15.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.904+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b019f900 con 0x7f37b0102980 2024-04-25T00:18:15.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.904+0000 7f37a77fe640 1 -- 172.21.15.110:0/1269271007 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a000f620 con 0x7f37b0102980 2024-04-25T00:18:15.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.904+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b019fea0 con 0x7f37b0102980 2024-04-25T00:18:15.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.905+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b0108bf0 con 0x7f37b0102980 2024-04-25T00:18:15.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.906+0000 7f37a77fe640 1 -- 172.21.15.110:0/1269271007 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f37a00109a0 con 0x7f37b0102980 2024-04-25T00:18:15.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.907+0000 7f37a77fe640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f379c0a2c10 0x7f379c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:15.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.907+0000 7f37b61f9640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f379c0a2c10 0x7f379c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:15.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.907+0000 7f37a77fe640 1 -- 172.21.15.110:0/1269271007 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f37a00cec90 con 0x7f37b0102980 2024-04-25T00:18:15.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.907+0000 7f37b61f9640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f379c0a2c10 0x7f379c0a50d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:15.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:15.911+0000 7f37a77fe640 1 -- 172.21.15.110:0/1269271007 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f37a0009de0 con 0x7f37b0102980 2024-04-25T00:18:16.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.092+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f37b0062340 con 0x7f37b0102980 2024-04-25T00:18:16.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.092+0000 7f37a77fe640 1 -- 172.21.15.110:0/1269271007 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f37a009bc20 con 0x7f37b0102980 2024-04-25T00:18:16.093 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T00:18:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f379c0a2c10 msgr2=0x7f379c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f379c0a2c10 0x7f379c0a50d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f37ac006010 tx=0x7f37ac000f40 comp rx=0 tx=0).stop 2024-04-25T00:18:16.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0102980 msgr2=0x7f37b0197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0102980 0x7f37b0197f30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f37a000d950 tx=0x7f37a000de10 comp rx=0 tx=0).stop 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 shutdown_connections 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f379c0a2c10 0x7f379c0a50d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37b0103340 0x7f37b019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37b0102980 0x7f37b0197f30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.094+0000 7f37b845d640 1 --2- 172.21.15.110:0/1269271007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37b0101790 0x7f37b01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.095+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 >> 172.21.15.110:0/1269271007 conn(0x7f37b00fcd20 msgr2=0x7f37b0105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.095+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 shutdown_connections 2024-04-25T00:18:16.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.095+0000 7f37b845d640 1 -- 172.21.15.110:0/1269271007 wait complete. 2024-04-25T00:18:16.105 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:15 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/544965646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:16.105 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:15 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2257424225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:16.106 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:18:16.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 -- 172.21.15.110:0/1366756954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c102990 msgr2=0x7fef3c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 --2- 172.21.15.110:0/1366756954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c102990 0x7fef3c102e10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fef30009900 tx=0x7fef3002a410 comp rx=0 tx=0).stop 2024-04-25T00:18:16.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 -- 172.21.15.110:0/1366756954 shutdown_connections 2024-04-25T00:18:16.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 --2- 172.21.15.110:0/1366756954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c103350 0x7fef3c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 --2- 172.21.15.110:0/1366756954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c102990 0x7fef3c102e10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 --2- 172.21.15.110:0/1366756954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef3c1017a0 0x7fef3c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 -- 172.21.15.110:0/1366756954 >> 172.21.15.110:0/1366756954 conn(0x7fef3c0fcd10 msgr2=0x7fef3c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:16.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 -- 172.21.15.110:0/1366756954 shutdown_connections 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.206+0000 7fef42790640 1 -- 172.21.15.110:0/1366756954 wait complete. 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef42790640 1 Processor -- start 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef42790640 1 -- start start 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef42790640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c1017a0 0x7fef3c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef42790640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef3c102990 0x7fef3c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef42790640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c103350 0x7fef3c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef42790640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c10d330 con 0x7fef3c1017a0 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef42790640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c10d1b0 con 0x7fef3c102990 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef42790640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c10d4b0 con 0x7fef3c103350 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef3bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c1017a0 0x7fef3c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef3bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c1017a0 0x7fef3c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60022/0 (socket says 172.21.15.110:60022) 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef3bfff640 1 -- 172.21.15.110:0/2695934611 learned_addr learned my addr 172.21.15.110:0/2695934611 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:16.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef40d2d640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c103350 0x7fef3c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.207+0000 7fef3b7fe640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef3c102990 0x7fef3c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef3bfff640 1 -- 172.21.15.110:0/2695934611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c103350 msgr2=0x7fef3c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef3bfff640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c103350 0x7fef3c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef3bfff640 1 -- 172.21.15.110:0/2695934611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef3c102990 msgr2=0x7fef3c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef3bfff640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef3c102990 0x7fef3c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef3bfff640 1 -- 172.21.15.110:0/2695934611 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3c19f650 con 0x7fef3c1017a0 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef3bfff640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c1017a0 0x7fef3c197a30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00d770 tx=0x7fef2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef397fa640 1 -- 172.21.15.110:0/2695934611 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c017070 con 0x7fef3c1017a0 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef397fa640 1 -- 172.21.15.110:0/2695934611 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef2c009c30 con 0x7fef3c1017a0 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3c19f940 con 0x7fef3c1017a0 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef397fa640 1 -- 172.21.15.110:0/2695934611 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c003d00 con 0x7fef3c1017a0 2024-04-25T00:18:16.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.208+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3c19fee0 con 0x7fef3c1017a0 2024-04-25T00:18:16.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.209+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef3c108b60 con 0x7fef3c1017a0 2024-04-25T00:18:16.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.210+0000 7fef397fa640 1 -- 172.21.15.110:0/2695934611 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fef2c003ea0 con 0x7fef3c1017a0 2024-04-25T00:18:16.212 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.210+0000 7fef397fa640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fef240a2b90 0x7fef240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.211+0000 7fef3b7fe640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fef240a2b90 0x7fef240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.211+0000 7fef397fa640 1 -- 172.21.15.110:0/2695934611 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fef2c0cecd0 con 0x7fef3c1017a0 2024-04-25T00:18:16.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.211+0000 7fef3b7fe640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fef240a2b90 0x7fef240a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fef30006010 tx=0x7fef30000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:16.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.215+0000 7fef397fa640 1 -- 172.21.15.110:0/2695934611 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fef2c009de0 con 0x7fef3c1017a0 2024-04-25T00:18:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:15 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/544965646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:15 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2257424225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:16.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:15 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/544965646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:16.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:15 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2257424225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:16.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.392+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef3c062340 con 0x7fef3c1017a0 2024-04-25T00:18:16.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.393+0000 7fef397fa640 1 -- 172.21.15.110:0/2695934611 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fef2c09b610 con 0x7fef3c1017a0 2024-04-25T00:18:16.393 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 64 2024-04-25T00:18:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fef240a2b90 msgr2=0x7fef240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fef240a2b90 0x7fef240a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fef30006010 tx=0x7fef30000bc0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c1017a0 msgr2=0x7fef3c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c1017a0 0x7fef3c197a30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00d770 tx=0x7fef2c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:18:16.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 shutdown_connections 2024-04-25T00:18:16.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fef240a2b90 0x7fef240a5050 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c103350 0x7fef3c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef3c102990 0x7fef3c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 --2- 172.21.15.110:0/2695934611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef3c1017a0 0x7fef3c197a30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 >> 172.21.15.110:0/2695934611 conn(0x7fef3c0fcd10 msgr2=0x7fef3c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:16.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 shutdown_connections 2024-04-25T00:18:16.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.395+0000 7fef42790640 1 -- 172.21.15.110:0/2695934611 wait complete. 2024-04-25T00:18:16.407 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- 172.21.15.110:0/1817783130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec103340 msgr2=0x7f15ec109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 --2- 172.21.15.110:0/1817783130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec103340 0x7f15ec109bd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009ca0 tx=0x7f15e402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- 172.21.15.110:0/1817783130 shutdown_connections 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 --2- 172.21.15.110:0/1817783130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec103340 0x7f15ec109bd0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 --2- 172.21.15.110:0/1817783130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15ec102980 0x7f15ec102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 --2- 172.21.15.110:0/1817783130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15ec101790 0x7f15ec101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- 172.21.15.110:0/1817783130 >> 172.21.15.110:0/1817783130 conn(0x7f15ec0fcd40 msgr2=0x7f15ec0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- 172.21.15.110:0/1817783130 shutdown_connections 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- 172.21.15.110:0/1817783130 wait complete. 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 Processor -- start 2024-04-25T00:18:16.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- start start 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec101790 0x7f15ec197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15ec102980 0x7f15ec197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15ec103340 0x7f15ec19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec10d2c0 con 0x7f15ec101790 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec10d140 con 0x7f15ec103340 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f401b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec10d440 con 0x7f15ec102980 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f1db7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec101790 0x7f15ec197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.504+0000 7f15f1db7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec101790 0x7f15ec197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40978/0 (socket says 172.21.15.110:40978) 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f1db7640 1 -- 172.21.15.110:0/1865606342 learned_addr learned my addr 172.21.15.110:0/1865606342 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f15b6640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15ec102980 0x7f15ec197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f1db7640 1 -- 172.21.15.110:0/1865606342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15ec102980 msgr2=0x7f15ec197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f1db7640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15ec102980 0x7f15ec197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f25b8640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15ec103340 0x7f15ec19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f1db7640 1 -- 172.21.15.110:0/1865606342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15ec103340 msgr2=0x7f15ec19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f1db7640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15ec103340 0x7f15ec19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f1db7640 1 -- 172.21.15.110:0/1865606342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec19f5b0 con 0x7f15ec101790 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f15b6640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15ec102980 0x7f15ec197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f1db7640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec101790 0x7f15ec197990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f15e000e9c0 tx=0x7f15e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15deffd640 1 -- 172.21.15.110:0/1865606342 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000cd80 con 0x7f15ec101790 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec19f8a0 con 0x7f15ec101790 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15deffd640 1 -- 172.21.15.110:0/1865606342 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15e00035f0 con 0x7f15ec101790 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15deffd640 1 -- 172.21.15.110:0/1865606342 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0010690 con 0x7f15ec101790 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.505+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec19fe40 con 0x7f15ec101790 2024-04-25T00:18:16.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.506+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec108b50 con 0x7f15ec101790 2024-04-25T00:18:16.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.508+0000 7f15deffd640 1 -- 172.21.15.110:0/1865606342 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f15e00037a0 con 0x7f15ec101790 2024-04-25T00:18:16.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.508+0000 7f15deffd640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f15d80a2b90 0x7f15d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.509+0000 7f15f15b6640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f15d80a2b90 0x7f15d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.509+0000 7f15deffd640 1 -- 172.21.15.110:0/1865606342 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f15e00cf5d0 con 0x7f15ec101790 2024-04-25T00:18:16.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.509+0000 7f15f15b6640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f15d80a2b90 0x7f15d80a5050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15ec198ea0 tx=0x7f15e8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:16.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.512+0000 7f15deffd640 1 -- 172.21.15.110:0/1865606342 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f15e009c560 con 0x7f15ec101790 2024-04-25T00:18:16.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.738+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f15ec02d170 con 0x7f15ec101790 2024-04-25T00:18:16.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.746+0000 7f15deffd640 1 -- 172.21.15.110:0/1865606342 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740375 (secure 0 0 0) 0x7f15e009be60 con 0x7f15ec101790 2024-04-25T00:18:16.757 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:16.757 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:18:16.757 INFO:teuthology.orchestra.run.smithi110.stdout: "epoch": 18, 2024-04-25T00:18:16.757 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.757 INFO:teuthology.orchestra.run.smithi110.stdout: "active_gid": 14152, 2024-04-25T00:18:16.757 INFO:teuthology.orchestra.run.smithi110.stdout: "active_name": "x", 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: "active_addrs": { 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:6800", 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 546485163 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v1", 2024-04-25T00:18:16.758 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:6801", 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 546485163 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "active_addr": "172.21.15.110:6801/546485163", 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "active_change": "2024-04-24T23:52:54.426213+0000", 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "available": true, 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "standbys": [ 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "gid": 24109, 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "z", 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:18:16.759 INFO:teuthology.orchestra.run.smithi110.stdout: "available_modules": [ 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "alerts", 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.760 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.761 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.762 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_destination": { 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_destination", 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:18:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_from_name": { 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_from_name", 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "Ceph", 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email From: name", 2024-04-25T00:18:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_host": { 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_host", 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP server", 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_password": { 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_password", 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Password to authenticate with", 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_port": { 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_port", 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "465", 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP port", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_sender": { 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_sender", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:18:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_ssl": { 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_ssl", 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_user": { 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_user", 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "User to authenticate as", 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "balancer", 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "active": { 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "active", 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_time": { 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_time", 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0000", 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:18:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_weekday": { 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_weekday", 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_max_iterations": { 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "25", 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "250", 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_metrics": { 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_metrics", 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_step": { 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_step", 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.5", 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0.001", 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "0.999", 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "end_time": { 2024-04-25T00:18:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_time", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2359", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "end_weekday": { 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_weekday", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "min_score": { 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_score", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "upmap", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "crush-compat", 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "none", 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "read", 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap", 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap-read" 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Balancer mode", 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_ids": { 2024-04-25T00:18:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_ids", 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:18:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_deviation": { 2024-04-25T00:18:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_deviation", 2024-04-25T00:18:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_optimizations": { 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm", 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_down_multiplier": { 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_down_multiplier", 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_refresh_rate": { 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_refresh_rate", 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "20", 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_starting_port": { 2024-04-25T00:18:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_starting_port", 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4721", 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_ptrace": { 2024-04-25T00:18:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_ptrace", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_interval": { 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_interval", 2024-04-25T00:18:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.7", 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm_log_destination": { 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm_log_destination", 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "file", 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "file,syslog", 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "syslog" 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "cgroups_split": { 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cgroups_split", 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "config_checks_enabled": { 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_checks_enabled", 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "config_dashboard": { 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_dashboard", 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:18:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_alertmanager": { 2024-04-25T00:18:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_alertmanager", 2024-04-25T00:18:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:18:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_base": { 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_base", 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_elasticsearch": { 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "elasticsearch container image", 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_grafana": { 2024-04-25T00:18:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_grafana", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_haproxy": { 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_haproxy", 2024-04-25T00:18:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "HAproxy container image", 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_agent": { 2024-04-25T00:18:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_collector": { 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:18:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_query": { 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger query container image", 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_keepalived": { 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_keepalived", 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Keepalived container image", 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_loki": { 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_loki", 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Loki container image", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_node_exporter": { 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_node_exporter", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_nvmeof": { 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_nvmeof", 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Nvme-of container image", 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_prometheus": { 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_prometheus", 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_promtail": { 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_promtail", 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Promtail container image", 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_samba": { 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_samba", 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_snmp_gateway": { 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "container_init": { 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_init", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "daemon_cache_timeout": { 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "900", 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "default_registry": { 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_registry", 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io", 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "device_cache_timeout": { 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_cache_timeout", 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1800", 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "device_enhanced_scan": { 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_enhanced_scan", 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "facts_cache_timeout": { 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "facts_cache_timeout", 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "grafana_dashboards_path": { 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "host_check_interval": { 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "host_check_interval", 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "hw_monitoring": { 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hw_monitoring", 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "inventory_list_all": { 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "inventory_list_all", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "log_refresh_metadata": { 2024-04-25T00:18:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_refresh_metadata", 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "*", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "max_count_per_host": { 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_count_per_host", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "max_osd_draining_count": { 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_osd_draining_count", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "migration_current": { 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "migration_current", 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "internal - do not modify", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "root", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm-package", 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "root" 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "oob_default_addr": { 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "oob_default_addr", 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "169.254.1.1", 2024-04-25T00:18:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "prometheus_alerts_path": { 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_insecure": { 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_insecure", 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_password": { 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_password", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_url": { 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_url", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_username": { 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_username", 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.853 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "secure_monitoring_stack": { 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.854 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "service_discovery_port": { 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "service_discovery_port", 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8765", 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.855 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "ssh_config_file": { 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssh_config_file", 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.856 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "use_agent": { 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_agent", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.857 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "use_repo_digest": { 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_repo_digest", 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.858 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_failed_host_check": { 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.859 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_daemons": { 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.860 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_hosts": { 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.861 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crash", 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.862 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.863 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.864 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.865 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.866 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "retain_interval": { 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retain_interval", 2024-04-25T00:18:16.867 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "31536000", 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_recent_interval": { 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_recent_interval", 2024-04-25T00:18:16.868 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1209600", 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.869 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dashboard", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.870 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.871 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.874 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.875 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.876 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.877 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.878 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.879 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.880 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.881 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.882 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.883 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.884 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.885 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_URL": { 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.886 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.887 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.888 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.889 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:18:16.890 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:18:16.891 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.891 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.891 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.891 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.891 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.892 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.892 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.892 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.892 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.892 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.892 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.892 INFO:teuthology.orchestra.run.smithi110.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:18:16.893 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:18:16.893 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.893 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.893 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.893 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.893 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.894 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.895 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.895 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.895 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.895 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.895 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.895 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.895 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.896 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.896 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:18:16.896 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:18:16.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.897 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.897 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.897 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.897 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.897 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.897 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.898 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.898 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.898 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.898 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.899 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.900 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.904 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:18:16.905 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.907 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8", 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "45", 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.912 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.913 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.914 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.916 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.917 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "cross_origin_url": { 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cross_origin_url", 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.918 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "crt_file": { 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crt_file", 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "debug": { 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "debug", 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:18:16.920 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "jwt_token_ttl": { 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "jwt_token_ttl", 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "28800", 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.921 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.922 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.923 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.924 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.926 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "motd": { 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "motd", 2024-04-25T00:18:16.927 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "The message of the day", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.928 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-04-25T00:18:16.929 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8080", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.930 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl_server_port": { 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl_server_port", 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8443", 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "redirect", 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect" 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.933 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.934 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "url_prefix": { 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url_prefix", 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "devicehealth", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_monitoring": { 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_monitoring", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "monitor device health metrics", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.936 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.937 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.938 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.939 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.940 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "mark_out_threshold": { 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mark_out_threshold", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2419200", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_name": { 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_name", 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "device_health_metrics", 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.942 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "retention_period": { 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retention_period", 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15552000", 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.943 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_frequency": { 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_frequency", 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.944 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "self_heal": { 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "self_heal", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.946 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_threshold": { 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_threshold", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "7257600", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.947 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "diskprediction_local", 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.948 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.949 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.950 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.951 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.952 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "predict_interval": { 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predict_interval", 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "predictor_model": { 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predictor_model", 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "prophetstor", 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "influx", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": false, 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "batch_size": { 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "batch_size", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5000", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:18:16.956 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "database": { 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "database", 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "ceph", 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.957 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "hostname": { 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hostname", 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "30", 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "5", 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.960 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: "password": { 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "password", 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi110.stdout: "port": { 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "port", 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8086", 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server port", 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "false", 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.968 INFO:teuthology.orchestra.run.smithi110.stdout: "threads": { 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threads", 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "32", 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "username": { 2024-04-25T00:18:16.969 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "username", 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "verify_ssl": { 2024-04-25T00:18:16.970 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "verify_ssl", 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "true", 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "insights", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.972 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.973 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.975 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "iostat", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.977 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.978 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.979 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.980 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.981 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "k8sevents", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_event_retention_days": { 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "7", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.982 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "config_check_secs": { 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_check_secs", 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "10", 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.983 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.984 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.985 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.986 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.987 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "localpool", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "failure_domain": { 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "failure_domain", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "host", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:18:16.989 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.990 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.991 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.992 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:16.993 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "min_size": { 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_size", 2024-04-25T00:18:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "num_rep": { 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "num_rep", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.995 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3", 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_num": { 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_num", 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.996 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "128", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "prefix": { 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prefix", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.997 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "subtree": { 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "subtree", 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "rack", 2024-04-25T00:18:16.998 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mds_autoscaler", 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:16.999 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.000 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.001 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.002 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.003 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.004 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirroring", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.005 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.006 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.007 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "nfs", 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.010 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.013 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.014 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "fail_fs": { 2024-04-25T00:18:17.015 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "fail_fs", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.016 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.019 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.020 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator": { 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "rook", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "test_orchestrator" 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Orchestrator backend", 2024-04-25T00:18:17.021 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_perf_query", 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.022 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.023 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.024 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.025 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.026 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_support", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.029 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.030 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.031 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.032 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_autoscaler", 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.033 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.034 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.035 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.036 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.037 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.038 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "threshold": { 2024-04-25T00:18:17.039 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threshold", 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1.0", 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "scaling threshold", 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "progress", 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_pg_recovery_event": { 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.041 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-04-25T00:18:17.042 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-04-25T00:18:17.043 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.043 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.043 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.043 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.044 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.044 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.045 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.045 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.045 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.046 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.046 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.046 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.046 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.046 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.046 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.047 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.048 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.049 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.050 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "max_completed_events": { 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_completed_events", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "50", 2024-04-25T00:18:17.051 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.052 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus", 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.053 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "cache": { 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cache", 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "exclude_perf_counters": { 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "exclude_perf_counters", 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.056 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.057 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.058 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.058 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.058 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.058 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.058 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.058 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.059 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.059 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.059 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.059 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.059 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.059 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.060 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.060 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.060 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.060 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.060 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.060 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.061 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.062 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools": { 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools", 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.063 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "300", 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_interval": { 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_interval", 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15.0", 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "9283", 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "stale_cache_strategy": { 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stale_cache_strategy", 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "log", 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.068 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "default", 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "default", 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "error" 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-04-25T00:18:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_support", 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.072 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.073 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.074 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.076 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_snap_create": { 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.077 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.078 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "trash_purge_schedule": { 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "trash_purge_schedule", 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.079 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "restful", 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_auth": { 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_auth", 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.080 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.081 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.082 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.083 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.084 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.085 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.086 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.087 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.088 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rgw", 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.089 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.090 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.091 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.092 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.093 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rook", 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "storage_class": { 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "storage_class", 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "local", 2024-04-25T00:18:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "selftest", 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "roption1": { 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption1", 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "roption2": { 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption2", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "xyz", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption1": { 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption1", 2024-04-25T00:18:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption2": { 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption2", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption3": { 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption3", 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption4": { 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption4", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption5": { 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption5", 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption6": { 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption6", 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption7": { 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption7", 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "42", 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "testkey": { 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testkey", 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "testlkey": { 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testlkey", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "testnewline": { 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testnewline", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snap_schedule", 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_m_granularity": { 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_m_granularity", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "dump_on_update": { 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dump_on_update", 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stats", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "status", 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telegraf", 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "address": { 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "address", 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15", 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.143 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telemetry", 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_basic": { 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_basic", 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.144 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_crash": { 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_crash", 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.145 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_device": { 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_device", 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.146 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_ident": { 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_ident", 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.147 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_perf": { 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_perf", 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.148 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "contact": { 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "contact", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.149 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "description": { 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "description", 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.150 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "device_url": { 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_url", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.151 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.152 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "24", 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "8", 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.153 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "last_opt_revision": { 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "last_opt_revision", 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1", 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.154 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard": { 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.155 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard_description": { 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard_description", 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.156 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.157 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.158 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.159 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.161 INFO:teuthology.orchestra.run.smithi110.stdout: "organization": { 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "organization", 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.162 INFO:teuthology.orchestra.run.smithi110.stdout: "proxy": { 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "proxy", 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.163 INFO:teuthology.orchestra.run.smithi110.stdout: "url": { 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url", 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "test_orchestrator", 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.165 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.166 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.167 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.168 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.170 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.171 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.172 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.174 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_clones": { 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_clones", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.175 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "periodic_async_work": { 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "periodic_async_work", 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.176 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Periodically check for async work", 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_delay": { 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.177 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.178 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix", 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "discovery_interval": { 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "discovery_interval", 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.179 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "100", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "identifier": { 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "identifier", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.180 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.181 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.182 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.184 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.185 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.186 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_host": { 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_host", 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.187 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_port": { 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_port", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10051", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.188 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_sender": { 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_sender", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.189 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: "gid": 24113, 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "y", 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: "available_modules": [ 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.190 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "alerts", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:18:17.191 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.192 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.194 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.196 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_destination": { 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_destination", 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.197 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_from_name": { 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_from_name", 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "Ceph", 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email From: name", 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_host": { 2024-04-25T00:18:17.198 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_host", 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP server", 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_password": { 2024-04-25T00:18:17.199 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_password", 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Password to authenticate with", 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_port": { 2024-04-25T00:18:17.200 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_port", 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "465", 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP port", 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_sender": { 2024-04-25T00:18:17.201 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_sender", 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_ssl": { 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_ssl", 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.203 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_user": { 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_user", 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "User to authenticate as", 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.204 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "balancer", 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "active": { 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "active", 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.205 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_time": { 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_time", 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0000", 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.206 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_weekday": { 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_weekday", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-04-25T00:18:17.207 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_max_iterations": { 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "25", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "250", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:18:17.208 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_metrics": { 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_metrics", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:18:17.209 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_step": { 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_step", 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.5", 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0.001", 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "0.999", 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.210 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "end_time": { 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_time", 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2359", 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "end_weekday": { 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_weekday", 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.213 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.214 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.215 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.216 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "min_score": { 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_score", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.217 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "upmap", 2024-04-25T00:18:17.218 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "crush-compat", 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "none", 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "read", 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap", 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap-read" 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Balancer mode", 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_ids": { 2024-04-25T00:18:17.219 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_ids", 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.221 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_deviation": { 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_deviation", 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.222 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_optimizations": { 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.223 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm", 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_down_multiplier": { 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_down_multiplier", 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.224 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_refresh_rate": { 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_refresh_rate", 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.225 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "20", 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_starting_port": { 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_starting_port", 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.226 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4721", 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_ptrace": { 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_ptrace", 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.227 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_interval": { 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_interval", 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.228 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.7", 2024-04-25T00:18:17.229 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm_log_destination": { 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm_log_destination", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.230 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "file", 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "file,syslog", 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "syslog" 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "cgroups_split": { 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cgroups_split", 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.231 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "config_checks_enabled": { 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_checks_enabled", 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.232 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "config_dashboard": { 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_dashboard", 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.233 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_alertmanager": { 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_alertmanager", 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.234 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_base": { 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_base", 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.235 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_elasticsearch": { 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.236 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "elasticsearch container image", 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_grafana": { 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_grafana", 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.237 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_haproxy": { 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_haproxy", 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.238 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "HAproxy container image", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_agent": { 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:18:17.239 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_collector": { 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.240 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_query": { 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.241 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger query container image", 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_keepalived": { 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_keepalived", 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.242 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Keepalived container image", 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_loki": { 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_loki", 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.243 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Loki container image", 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_node_exporter": { 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_node_exporter", 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.244 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_nvmeof": { 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_nvmeof", 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.245 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Nvme-of container image", 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_prometheus": { 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_prometheus", 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.246 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_promtail": { 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_promtail", 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.247 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Promtail container image", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_samba": { 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_samba", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:18:17.248 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_snmp_gateway": { 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:18:17.249 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "container_init": { 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_init", 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.250 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "daemon_cache_timeout": { 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.251 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:18:17.252 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "900", 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.253 INFO:teuthology.orchestra.run.smithi110.stdout: "default_registry": { 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_registry", 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io", 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:18:17.254 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "device_cache_timeout": { 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_cache_timeout", 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1800", 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.255 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: "device_enhanced_scan": { 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_enhanced_scan", 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.256 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "facts_cache_timeout": { 2024-04-25T00:18:17.257 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "facts_cache_timeout", 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.258 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "grafana_dashboards_path": { 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.259 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "host_check_interval": { 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "host_check_interval", 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.260 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.261 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.262 INFO:teuthology.orchestra.run.smithi110.stdout: "hw_monitoring": { 2024-04-25T00:18:17.262 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hw_monitoring", 2024-04-25T00:18:17.262 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.262 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.262 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.263 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.263 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.263 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.263 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.263 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:18:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: "inventory_list_all": { 2024-04-25T00:18:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "inventory_list_all", 2024-04-25T00:18:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.264 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.266 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.267 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "log_refresh_metadata": { 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_refresh_metadata", 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.269 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.270 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.271 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.272 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.273 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "*", 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.274 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "max_count_per_host": { 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_count_per_host", 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.275 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "max_osd_draining_count": { 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_osd_draining_count", 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "migration_current": { 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "migration_current", 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "internal - do not modify", 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "root", 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm-package", 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "root" 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:18:17.278 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "oob_default_addr": { 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "oob_default_addr", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "169.254.1.1", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.279 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "prometheus_alerts_path": { 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.280 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_insecure": { 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_insecure", 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.281 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_password": { 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_password", 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.282 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_url": { 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_url", 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.283 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_username": { 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_username", 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.284 INFO:teuthology.orchestra.run.smithi110.stdout: "secure_monitoring_stack": { 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "service_discovery_port": { 2024-04-25T00:18:17.285 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "service_discovery_port", 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8765", 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.286 INFO:teuthology.orchestra.run.smithi110.stdout: "ssh_config_file": { 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssh_config_file", 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "use_agent": { 2024-04-25T00:18:17.287 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_agent", 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "use_repo_digest": { 2024-04-25T00:18:17.288 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_repo_digest", 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_failed_host_check": { 2024-04-25T00:18:17.289 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_daemons": { 2024-04-25T00:18:17.290 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.291 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_hosts": { 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.292 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crash", 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.293 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.294 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.295 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.296 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.297 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "retain_interval": { 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retain_interval", 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "31536000", 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.298 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_recent_interval": { 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_recent_interval", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1209600", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:18:17.299 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dashboard", 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.300 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:18:17.301 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.304 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.305 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.306 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.307 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.308 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.309 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.310 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.311 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.312 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:18:17.313 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:18:17.314 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.315 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:18:17.316 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_URL": { 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:18:17.317 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:18:17.318 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.319 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:18:17.320 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.321 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.322 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.323 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.324 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.325 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:18:17.326 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:18:17.327 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:18:17.328 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:18:17.329 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:18:17.330 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:18:17.331 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.332 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.334 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.335 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8", 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.336 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "45", 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.337 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.338 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.339 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.340 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.341 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.342 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.343 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.344 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "cross_origin_url": { 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cross_origin_url", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.345 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "crt_file": { 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crt_file", 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.346 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "debug": { 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "debug", 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.347 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "jwt_token_ttl": { 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "jwt_token_ttl", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "28800", 2024-04-25T00:18:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.349 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.350 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.351 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.352 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.353 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.354 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "motd": { 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "motd", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.355 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "The message of the day", 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.356 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-04-25T00:18:17.357 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8080", 2024-04-25T00:18:17.358 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.359 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl_server_port": { 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl_server_port", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8443", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.360 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "redirect", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.361 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect" 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.362 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "url_prefix": { 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url_prefix", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.363 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "devicehealth", 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.364 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_monitoring": { 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_monitoring", 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "monitor device health metrics", 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.365 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.366 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.367 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.368 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.369 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.370 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "mark_out_threshold": { 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mark_out_threshold", 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2419200", 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.371 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_name": { 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_name", 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "device_health_metrics", 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.372 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "retention_period": { 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retention_period", 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15552000", 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.373 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_frequency": { 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_frequency", 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.374 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "self_heal": { 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "self_heal", 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.375 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:18:17.376 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_threshold": { 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_threshold", 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "7257600", 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:18:17.377 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "diskprediction_local", 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.378 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.379 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.380 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.381 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.382 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.383 INFO:teuthology.orchestra.run.smithi110.stdout: "predict_interval": { 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predict_interval", 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.384 INFO:teuthology.orchestra.run.smithi110.stdout: "predictor_model": { 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predictor_model", 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "prophetstor", 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.385 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "influx", 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": false, 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "batch_size": { 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "batch_size", 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5000", 2024-04-25T00:18:17.387 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "database": { 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "database", 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.388 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "ceph", 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "hostname": { 2024-04-25T00:18:17.389 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hostname", 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "30", 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "5", 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.392 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.393 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.394 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.395 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.396 INFO:teuthology.orchestra.run.smithi110.stdout: "password": { 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "password", 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "port": { 2024-04-25T00:18:17.397 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "port", 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8086", 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server port", 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-04-25T00:18:17.398 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "false", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "threads": { 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threads", 2024-04-25T00:18:17.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "32", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "username": { 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "username", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.400 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "verify_ssl": { 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "verify_ssl", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "true", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.401 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "insights", 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.402 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.403 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.404 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.405 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.406 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "iostat", 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.407 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.408 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.409 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.410 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.411 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "k8sevents", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_event_retention_days": { 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "7", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:18:17.413 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "config_check_secs": { 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_check_secs", 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "10", 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:18:17.414 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.415 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.416 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.417 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.418 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "localpool", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "failure_domain": { 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "failure_domain", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "host", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:18:17.420 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.421 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.422 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.423 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.424 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.425 INFO:teuthology.orchestra.run.smithi110.stdout: "min_size": { 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_size", 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.426 INFO:teuthology.orchestra.run.smithi110.stdout: "num_rep": { 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "num_rep", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_num": { 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_num", 2024-04-25T00:18:17.427 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "128", 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.428 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "prefix": { 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prefix", 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.429 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "subtree": { 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "subtree", 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "rack", 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:18:17.430 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mds_autoscaler", 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.432 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.433 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.434 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.435 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirroring", 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.437 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.438 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.439 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.440 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.441 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "nfs", 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.442 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.443 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.444 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.445 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.446 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.447 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "fail_fs": { 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "fail_fs", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.448 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.449 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.450 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.451 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.452 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.453 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator": { 2024-04-25T00:18:17.454 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-04-25T00:18:17.454 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.454 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.454 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.454 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.454 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.454 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.454 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.455 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm", 2024-04-25T00:18:17.455 INFO:teuthology.orchestra.run.smithi110.stdout: "rook", 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: "test_orchestrator" 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Orchestrator backend", 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_perf_query", 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.457 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.458 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.459 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.460 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.461 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.462 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_support", 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.463 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.464 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.465 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.465 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.465 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.465 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.465 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.465 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.466 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.467 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.468 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.469 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_autoscaler", 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.470 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.472 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.473 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.474 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.475 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.476 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "threshold": { 2024-04-25T00:18:17.477 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threshold", 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1.0", 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "scaling threshold", 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "progress", 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_pg_recovery_event": { 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.479 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.480 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.481 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.482 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.483 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.484 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.485 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.486 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "max_completed_events": { 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_completed_events", 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.487 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "50", 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.488 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.489 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus", 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "cache": { 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cache", 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.490 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "exclude_perf_counters": { 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "exclude_perf_counters", 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.491 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.492 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.494 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.495 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.496 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools": { 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools", 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.498 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "300", 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.499 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_interval": { 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_interval", 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15.0", 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.500 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-04-25T00:18:17.501 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.502 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "9283", 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "stale_cache_strategy": { 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stale_cache_strategy", 2024-04-25T00:18:17.503 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "log", 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "default", 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "default", 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: "error" 2024-04-25T00:18:17.505 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-04-25T00:18:17.506 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_support", 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.507 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.508 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.509 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.510 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.511 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.512 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_snap_create": { 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.513 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.514 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "trash_purge_schedule": { 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "trash_purge_schedule", 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.515 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "restful", 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_auth": { 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_auth", 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.517 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.519 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.520 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.521 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.522 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.523 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.524 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.525 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:17.526 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.527 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rgw", 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.528 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.529 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.530 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.531 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.532 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rook", 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.534 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.535 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.536 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.537 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "storage_class": { 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "storage_class", 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "local", 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.540 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "selftest", 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.541 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.542 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.543 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.544 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.545 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.546 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "roption1": { 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption1", 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.547 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "roption2": { 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption2", 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "xyz", 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.548 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption1": { 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption1", 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.549 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption2": { 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption2", 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption3": { 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption3", 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.551 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption4": { 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption4", 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.552 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption5": { 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption5", 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.553 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption6": { 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption6", 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.554 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption7": { 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption7", 2024-04-25T00:18:17.555 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "42", 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "testkey": { 2024-04-25T00:18:17.556 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testkey", 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "testlkey": { 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testlkey", 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.558 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "testnewline": { 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testnewline", 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.559 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snap_schedule", 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_m_granularity": { 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_m_granularity", 2024-04-25T00:18:17.560 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "dump_on_update": { 2024-04-25T00:18:17.561 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dump_on_update", 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.562 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.563 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.564 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.566 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.567 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stats", 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.568 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.569 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.570 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.571 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.572 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.573 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "status", 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.574 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.575 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.576 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.577 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.578 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telegraf", 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "address": { 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "address", 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.580 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15", 2024-04-25T00:18:17.581 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.582 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.583 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.584 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.585 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.587 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telemetry", 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_basic": { 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_basic", 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.588 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.589 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_crash": { 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_crash", 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.590 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_device": { 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_device", 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.591 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_ident": { 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_ident", 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.592 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_perf": { 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_perf", 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.593 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:18:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "contact": { 2024-04-25T00:18:17.594 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "contact", 2024-04-25T00:18:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.595 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.596 INFO:teuthology.orchestra.run.smithi110.stdout: "description": { 2024-04-25T00:18:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "description", 2024-04-25T00:18:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.597 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.598 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.598 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.598 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.598 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.598 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.598 INFO:teuthology.orchestra.run.smithi110.stdout: "device_url": { 2024-04-25T00:18:17.599 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_url", 2024-04-25T00:18:17.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.599 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.599 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:18:17.599 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.599 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.599 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.600 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "24", 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "8", 2024-04-25T00:18:17.601 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "last_opt_revision": { 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "last_opt_revision", 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.602 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1", 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard": { 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard", 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.603 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard_description": { 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard_description", 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.605 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.606 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.607 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.608 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.609 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "organization": { 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "organization", 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.611 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "proxy": { 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "proxy", 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.612 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "url": { 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url", 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.613 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "test_orchestrator", 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.614 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.615 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.616 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.617 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.618 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.619 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-04-25T00:18:17.620 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.621 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.622 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.623 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.624 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.624 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.624 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.624 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.624 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.624 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.624 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.625 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.625 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.626 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.627 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_clones": { 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_clones", 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4", 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:18:17.628 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "periodic_async_work": { 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "periodic_async_work", 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.629 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Periodically check for async work", 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_delay": { 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.631 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix", 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "discovery_interval": { 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "discovery_interval", 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "100", 2024-04-25T00:18:17.633 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "identifier": { 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "identifier", 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.634 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.635 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.636 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.637 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.638 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.639 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.640 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.641 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_host": { 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_host", 2024-04-25T00:18:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_port": { 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_port", 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10051", 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.644 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_sender": { 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_sender", 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.645 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: "modules": [ 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm", 2024-04-25T00:18:17.646 INFO:teuthology.orchestra.run.smithi110.stdout: "dashboard", 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "iostat", 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "nfs", 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "restful" 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "available_modules": [ 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "alerts", 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.647 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:18:17.648 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.649 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.650 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.651 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.653 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.654 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_destination": { 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_destination", 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_from_name": { 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_from_name", 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.656 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "Ceph", 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Email From: name", 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_host": { 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_host", 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.657 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP server", 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_password": { 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_password", 2024-04-25T00:18:17.658 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Password to authenticate with", 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.659 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_port": { 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_port", 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "465", 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP port", 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_sender": { 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_sender", 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.661 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_ssl": { 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_ssl", 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.662 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "smtp_user": { 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "smtp_user", 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.663 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "User to authenticate as", 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "balancer", 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.664 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "active": { 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "active", 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.665 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_time": { 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_time", 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0000", 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.666 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "begin_weekday": { 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "begin_weekday", 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-04-25T00:18:17.667 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_max_iterations": { 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "25", 2024-04-25T00:18:17.668 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "250", 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_metrics": { 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_metrics", 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.669 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "crush_compat_step": { 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crush_compat_step", 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.670 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.5", 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0.001", 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "0.999", 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "end_time": { 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_time", 2024-04-25T00:18:17.671 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2359", 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.672 INFO:teuthology.orchestra.run.smithi110.stdout: "end_weekday": { 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "end_weekday", 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "0", 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "6", 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.674 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.675 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.676 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.677 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.678 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "min_score": { 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_score", 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.679 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "upmap", 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.680 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "crush-compat", 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "none", 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "read", 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap", 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap-read" 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Balancer mode", 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_ids": { 2024-04-25T00:18:17.681 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_ids", 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:18:17.683 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_deviation": { 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_deviation", 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.684 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "upmap_max_optimizations": { 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.685 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm", 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.686 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_down_multiplier": { 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_down_multiplier", 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_refresh_rate": { 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_refresh_rate", 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "20", 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.688 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "agent_starting_port": { 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "agent_starting_port", 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4721", 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.689 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_ptrace": { 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_ptrace", 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:18:17.690 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_interval": { 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_interval", 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.691 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0.7", 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.692 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm_log_destination": { 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cephadm_log_destination", 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.693 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "file", 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "file,syslog", 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "syslog" 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "cgroups_split": { 2024-04-25T00:18:17.694 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cgroups_split", 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.695 INFO:teuthology.orchestra.run.smithi110.stdout: "config_checks_enabled": { 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_checks_enabled", 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.696 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "config_dashboard": { 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_dashboard", 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:18:17.697 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_alertmanager": { 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_alertmanager", 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.698 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_base": { 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_base", 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.699 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_elasticsearch": { 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.700 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "elasticsearch container image", 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_grafana": { 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_grafana", 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.701 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.702 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_haproxy": { 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_haproxy", 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "HAproxy container image", 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.703 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_agent": { 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:18:17.704 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_collector": { 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.705 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_jaeger_query": { 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.706 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Jaeger query container image", 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_keepalived": { 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_keepalived", 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.707 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Keepalived container image", 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_loki": { 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_loki", 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.708 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Loki container image", 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.709 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_node_exporter": { 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_node_exporter", 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.710 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_nvmeof": { 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_nvmeof", 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Nvme-of container image", 2024-04-25T00:18:17.711 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_prometheus": { 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_prometheus", 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.712 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Prometheus container image", 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_promtail": { 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_promtail", 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.713 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Promtail container image", 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_samba": { 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_samba", 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.714 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "container_image_snmp_gateway": { 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.715 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "container_init": { 2024-04-25T00:18:17.716 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "container_init", 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "daemon_cache_timeout": { 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.718 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "900", 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "default_registry": { 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "default_registry", 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "docker.io", 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:18:17.720 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "device_cache_timeout": { 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_cache_timeout", 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1800", 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.721 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "device_enhanced_scan": { 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_enhanced_scan", 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "facts_cache_timeout": { 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "facts_cache_timeout", 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.723 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "grafana_dashboards_path": { 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.724 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "host_check_interval": { 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "host_check_interval", 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.725 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "hw_monitoring": { 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hw_monitoring", 2024-04-25T00:18:17.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.727 INFO:teuthology.orchestra.run.smithi110.stdout: "inventory_list_all": { 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "inventory_list_all", 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.728 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.729 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "log_refresh_metadata": { 2024-04-25T00:18:17.730 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_refresh_metadata", 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.732 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.733 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.734 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.735 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.736 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "*", 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.737 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "max_count_per_host": { 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_count_per_host", 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.738 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "max_osd_draining_count": { 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_osd_draining_count", 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.739 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "migration_current": { 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "migration_current", 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.740 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "internal - do not modify", 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "mode": { 2024-04-25T00:18:17.741 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mode", 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "root", 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm-package", 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "root" 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.742 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "oob_default_addr": { 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "oob_default_addr", 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "169.254.1.1", 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.743 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "prometheus_alerts_path": { 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.744 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_insecure": { 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_insecure", 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.745 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_password": { 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_password", 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.746 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_url": { 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_url", 2024-04-25T00:18:17.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "registry_username": { 2024-04-25T00:18:17.748 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "registry_username", 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "secure_monitoring_stack": { 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.750 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "service_discovery_port": { 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "service_discovery_port", 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8765", 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.751 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "ssh_config_file": { 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssh_config_file", 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.752 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "use_agent": { 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_agent", 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.753 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "use_repo_digest": { 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "use_repo_digest", 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.754 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_failed_host_check": { 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:18:17.755 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.756 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_daemons": { 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.757 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_on_stray_hosts": { 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:18:17.758 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crash", 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.759 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.760 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.761 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.762 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.763 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.764 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "retain_interval": { 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retain_interval", 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.765 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "31536000", 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_recent_interval": { 2024-04-25T00:18:17.766 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_recent_interval", 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1209600", 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dashboard", 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.768 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:18:17.769 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.770 INFO:teuthology.orchestra.run.smithi110.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.771 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.772 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.773 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.774 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.775 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.776 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:18:17.777 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.779 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.780 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.781 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.782 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.783 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.784 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:18:17.785 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_URL": { 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.787 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.788 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.789 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.790 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.791 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.792 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:18:17.793 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.795 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.802 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.805 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.806 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8", 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:18:17.808 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "45", 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:18:17.809 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "admin", 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.811 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.812 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.813 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:17.814 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.815 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.815 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.815 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.815 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.815 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.815 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.815 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:18:17.816 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:18:17.816 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.816 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.816 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.816 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.816 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.816 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.816 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.817 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.818 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.818 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.818 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.818 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.818 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.818 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.818 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.819 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.819 INFO:teuthology.orchestra.run.smithi110.stdout: "cross_origin_url": { 2024-04-25T00:18:17.819 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cross_origin_url", 2024-04-25T00:18:17.819 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.820 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.820 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.820 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.820 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.820 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.821 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.821 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.821 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.821 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.821 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.822 INFO:teuthology.orchestra.run.smithi110.stdout: "crt_file": { 2024-04-25T00:18:17.822 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "crt_file", 2024-04-25T00:18:17.822 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.822 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.822 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.823 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.823 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.823 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.823 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.823 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.824 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.824 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.824 INFO:teuthology.orchestra.run.smithi110.stdout: "debug": { 2024-04-25T00:18:17.825 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "debug", 2024-04-25T00:18:17.825 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.825 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.825 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.825 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.825 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.826 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.826 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.826 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:18:17.826 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.826 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.827 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.827 INFO:teuthology.orchestra.run.smithi110.stdout: "jwt_token_ttl": { 2024-04-25T00:18:17.827 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "jwt_token_ttl", 2024-04-25T00:18:17.827 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.827 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.828 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "28800", 2024-04-25T00:18:17.829 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.830 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.830 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.830 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.830 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.830 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.831 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.831 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-04-25T00:18:17.831 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-04-25T00:18:17.831 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.831 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.831 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.831 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.832 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.833 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.834 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.836 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.837 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "motd": { 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "motd", 2024-04-25T00:18:17.838 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "The message of the day", 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.839 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.840 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.841 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8080", 2024-04-25T00:18:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl_server_port": { 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl_server_port", 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8443", 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-04-25T00:18:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "redirect", 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "redirect" 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-04-25T00:18:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "url_prefix": { 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url_prefix", 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "devicehealth", 2024-04-25T00:18:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_monitoring": { 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_monitoring", 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "monitor device health metrics", 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "mark_out_threshold": { 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mark_out_threshold", 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "2419200", 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "pool_name": { 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pool_name", 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "device_health_metrics", 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "retention_period": { 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "retention_period", 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15552000", 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_frequency": { 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_frequency", 2024-04-25T00:18:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "self_heal": { 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "self_heal", 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "warn_threshold": { 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "warn_threshold", 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "7257600", 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "diskprediction_local", 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "predict_interval": { 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predict_interval", 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "86400", 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "predictor_model": { 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "predictor_model", 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "prophetstor", 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "600", 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "influx", 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": false, 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "batch_size": { 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "batch_size", 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5000", 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "database": { 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "database", 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "ceph", 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "hostname": { 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "hostname", 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "30", 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "5", 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "password": { 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "password", 2024-04-25T00:18:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "port": { 2024-04-25T00:18:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "port", 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "8086", 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "InfluxDB server port", 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "ssl": { 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ssl", 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "false", 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "threads": { 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threads", 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "32", 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "username": { 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "username", 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "verify_ssl": { 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "verify_ssl", 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "true", 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "insights", 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "iostat", 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "k8sevents", 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_event_retention_days": { 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "7", 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:18:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "config_check_secs": { 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "config_check_secs", 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "10", 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "localpool", 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "failure_domain": { 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "failure_domain", 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "host", 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "min_size": { 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "min_size", 2024-04-25T00:18:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "num_rep": { 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "num_rep", 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3", 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_num": { 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_num", 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "128", 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:18:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "prefix": { 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prefix", 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "subtree": { 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "subtree", 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "rack", 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mds_autoscaler", 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirroring", 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.938 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.939 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.940 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "nfs", 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.941 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.942 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.943 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.944 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.945 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.946 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.947 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "fail_fs": { 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "fail_fs", 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:18:17.948 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.949 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.950 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.951 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.952 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.953 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator": { 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "orchestrator", 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "cephadm", 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "rook", 2024-04-25T00:18:17.955 INFO:teuthology.orchestra.run.smithi110.stdout: "test_orchestrator" 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Orchestrator backend", 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_perf_query", 2024-04-25T00:18:17.956 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.957 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.958 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.960 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.961 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "osd_support", 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.963 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.964 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.965 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.966 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.967 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.968 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "pg_autoscaler", 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.970 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.971 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.972 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.973 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.974 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.975 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "threshold": { 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "threshold", 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "3.0", 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1.0", 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "scaling threshold", 2024-04-25T00:18:17.976 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "progress", 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_pg_recovery_event": { 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.977 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-04-25T00:18:17.978 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.979 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.980 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.981 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.982 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.983 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.984 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "max_completed_events": { 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_completed_events", 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "50", 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.985 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "sleep_interval": { 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "sleep_interval", 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "5", 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.986 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "prometheus", 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:17.987 INFO:teuthology.orchestra.run.smithi110.stdout: "cache": { 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "cache", 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.988 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "exclude_perf_counters": { 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "exclude_perf_counters", 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:18:17.989 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.990 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.992 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:17.993 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools": { 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools", 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "300", 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.997 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "scrape_interval": { 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "scrape_interval", 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15.0", 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:17.998 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "::", 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:17.999 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "9283", 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.000 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "stale_cache_strategy": { 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stale_cache_strategy", 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "log", 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.001 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_behaviour": { 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_behaviour", 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.002 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "default", 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "default", 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "error" 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.003 INFO:teuthology.orchestra.run.smithi110.stdout: "standby_error_status_code": { 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "standby_error_status_code", 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "500", 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "400", 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "599", 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.004 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_support", 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.005 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.006 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.007 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.008 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.009 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.010 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_snap_create": { 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10", 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.011 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.012 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "trash_purge_schedule": { 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "trash_purge_schedule", 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.013 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "restful", 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.014 INFO:teuthology.orchestra.run.smithi110.stdout: "enable_auth": { 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enable_auth", 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.015 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "key_file": { 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "key_file", 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.016 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.017 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.018 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.019 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.020 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "server_addr": { 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_addr", 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "server_port": { 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "server_port", 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rgw", 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rook", 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "storage_class": { 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "storage_class", 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "local", 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "selftest", 2024-04-25T00:18:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "roption1": { 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption1", 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "roption2": { 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "roption2", 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "xyz", 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption1": { 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption1", 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption2": { 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption2", 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption3": { 2024-04-25T00:18:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption3", 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "float", 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption4": { 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption4", 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption5": { 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption5", 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.051 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption6": { 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption6", 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.052 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "rwoption7": { 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rwoption7", 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.053 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "1", 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "42", 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "testkey": { 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testkey", 2024-04-25T00:18:18.054 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.055 INFO:teuthology.orchestra.run.smithi110.stdout: "testlkey": { 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testlkey", 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.056 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "testnewline": { 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "testnewline", 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.057 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snap_schedule", 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: "allow_m_granularity": { 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "allow_m_granularity", 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.059 INFO:teuthology.orchestra.run.smithi110.stdout: "dump_on_update": { 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "dump_on_update", 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.060 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.061 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.062 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.063 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.064 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.065 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "stats", 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.068 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.072 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "status", 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.073 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.074 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.075 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.076 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.077 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.078 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telegraf", 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.079 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "address": { 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "address", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.080 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "15", 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.081 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.082 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.083 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.084 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.085 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.086 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "telemetry", 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_basic": { 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_basic", 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.088 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_crash": { 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_crash", 2024-04-25T00:18:18.089 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.090 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_device": { 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_device", 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.091 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_ident": { 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_ident", 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.092 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "channel_perf": { 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "channel_perf", 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.093 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "contact": { 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "contact", 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.094 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "description": { 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "description", 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.095 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "device_url": { 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "device_url", 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.096 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "enabled": { 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "enabled", 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.097 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "24", 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "8", 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.099 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "last_opt_revision": { 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "last_opt_revision", 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "1", 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.100 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard": { 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard", 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.101 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "leaderboard_description": { 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "leaderboard_description", 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.102 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.103 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.104 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.105 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.106 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.107 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.108 INFO:teuthology.orchestra.run.smithi110.stdout: "organization": { 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "organization", 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "proxy": { 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "proxy", 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.110 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "url": { 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "url", 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.111 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "test_orchestrator", 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.112 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.113 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.114 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.116 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.117 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.118 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.119 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.120 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.121 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.122 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.123 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "max_concurrent_clones": { 2024-04-25T00:18:18.124 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "max_concurrent_clones", 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "4", 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.125 INFO:teuthology.orchestra.run.smithi110.stdout: "periodic_async_work": { 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "periodic_async_work", 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Periodically check for async work", 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.126 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_delay": { 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "0", 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.127 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "True", 2024-04-25T00:18:18.128 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix", 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "can_run": true, 2024-04-25T00:18:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: "error_string": "", 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "module_options": { 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "discovery_interval": { 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "discovery_interval", 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "uint", 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "100", 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.130 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "identifier": { 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "identifier", 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.131 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "interval": { 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "interval", 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "secs", 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "60", 2024-04-25T00:18:18.132 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "log_level": { 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_level", 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.133 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.134 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster": { 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster", 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.135 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_cluster_level": { 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_cluster_level", 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "info", 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.136 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [ 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "", 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "critical", 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "debug", 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "error", 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "info", 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "warning" 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "log_to_file": { 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "log_to_file", 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "bool", 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 1, 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "False", 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.138 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_host": { 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_host", 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "", 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.139 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_port": { 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_port", 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "int", 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "10051", 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.140 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "zabbix_sender": { 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "zabbix_sender", 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "str", 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "level": "advanced", 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flags": 0, 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "min": "", 2024-04-25T00:18:18.141 INFO:teuthology.orchestra.run.smithi110.stdout: "max": "", 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: "enum_allowed": [], 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: "desc": "", 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: "long_desc": "", 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: "tags": [], 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: "see_also": [] 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: "services": { 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: "dashboard": "https://172.21.15.110:8443/" 2024-04-25T00:18:18.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "always_on_modules": { 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "octopus": [ 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-04-25T00:18:18.143 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "pacific": [ 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-04-25T00:18:18.144 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "quincy": [ 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-04-25T00:18:18.145 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "reef": [ 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-04-25T00:18:18.146 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "squid": [ 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "balancer", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "devicehealth", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "orchestrator", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "pg_autoscaler", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "progress", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "rbd_support", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "status", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "telemetry", 2024-04-25T00:18:18.147 INFO:teuthology.orchestra.run.smithi110.stdout: "volumes" 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: "last_failure_osd_epoch": 3, 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: "active_clients": [ 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "devicehealth", 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 1380653271 2024-04-25T00:18:18.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "libcephsqlite", 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 2891513793 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:18.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "rbd_support", 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 4162407706 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.150 INFO:teuthology.orchestra.run.smithi110.stdout: "name": "volumes", 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: "addrvec": [ 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v2", 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "172.21.15.110:0", 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 135252951 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T00:18:18.151 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:18:18.153 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:16 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1269271007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:18.153 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:16 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2695934611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:18.153 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:16 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1865606342' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:18:18.153 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:16 smithi173 ceph-mon[32402]: pgmap v838: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-04-25T00:18:18.153 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:16 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1269271007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:18.153 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:16 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2695934611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:18.153 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:16 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1865606342' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:18:18.153 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:16 smithi188 ceph-mon[33285]: pgmap v838: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-04-25T00:18:18.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f15d80a2b90 msgr2=0x7f15d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:18.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f15d80a2b90 0x7f15d80a5050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15ec198ea0 tx=0x7f15e8005d30 comp rx=0 tx=0).stop 2024-04-25T00:18:18.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec101790 msgr2=0x7f15ec197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec101790 0x7f15ec197990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f15e000e9c0 tx=0x7f15e000ee80 comp rx=0 tx=0).stop 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 shutdown_connections 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f15d80a2b90 0x7f15d80a5050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15ec103340 0x7f15ec19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15ec102980 0x7f15ec197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 --2- 172.21.15.110:0/1865606342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15ec101790 0x7f15ec197990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 >> 172.21.15.110:0/1865606342 conn(0x7f15ec0fcd40 msgr2=0x7f15ec105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 shutdown_connections 2024-04-25T00:18:18.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:16.751+0000 7f15f401b640 1 -- 172.21.15.110:0/1865606342 wait complete. 2024-04-25T00:18:18.162 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T00:18:18.162 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T00:18:18.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:16 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1269271007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:18.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:16 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2695934611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:18.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:16 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1865606342' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:18:18.163 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:16 smithi110 ceph-mon[29187]: pgmap v838: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-04-25T00:18:18.453 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:19.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:18 smithi188 ceph-mon[33285]: pgmap v839: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:18:19.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:18 smithi110 ceph-mon[29187]: pgmap v839: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:18:19.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:18 smithi173 ceph-mon[32402]: pgmap v839: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4, 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 4, 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 2944, 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 448, 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.002088094, 2024-04-25T00:18:20.990 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000044044, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000735387, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000824422, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000274807 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T00:18:20.991 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 226, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 195168, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.016876405, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.012401585, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.001712037, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:20.992 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 224, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.010096150, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000045072 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 195168 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 298, 2024-04-25T00:18:20.993 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 295, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 501024, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 182160, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.039599031, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.015716873, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.016170510, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.006849030, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 294, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.009482753, 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000032254 2024-04-25T00:18:20.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 501024, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 182160 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:20.995 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-04-25T00:18:20.997 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148704, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 29, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:20.998 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 928, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:20.999 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 300, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 458927, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 300, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 300, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 458927, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:21.000 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:21.001 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:21.001 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:21.001 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:18:21.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:20 smithi188 ceph-mon[33285]: pgmap v840: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.6 MiB/s wr, 815 op/s 2024-04-25T00:18:21.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:20 smithi110 ceph-mon[29187]: pgmap v840: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.6 MiB/s wr, 815 op/s 2024-04-25T00:18:21.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:20 smithi173 ceph-mon[32402]: pgmap v840: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.6 MiB/s wr, 815 op/s 2024-04-25T00:18:21.903 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T00:18:21.903 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T00:18:21.903 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T00:18:21.904 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T00:18:22.151 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:23.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:22 smithi110 ceph-mon[29187]: pgmap v841: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.6 MiB/s wr, 745 op/s 2024-04-25T00:18:23.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:22 smithi188 ceph-mon[33285]: pgmap v841: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.6 MiB/s wr, 745 op/s 2024-04-25T00:18:23.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:22 smithi173 ceph-mon[32402]: pgmap v841: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.6 MiB/s wr, 745 op/s 2024-04-25T00:18:24.309 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 4, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 2944, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 448, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.001495500, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000029630, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000620947, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:24.310 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000690578, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000230192 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 230, 2024-04-25T00:18:24.311 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 195872, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.016062653, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.012401638, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.001625918, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 228, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.010999541, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000048243 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:24.312 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 195872 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 305, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 302, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 502992, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 186160, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.041120884, 2024-04-25T00:18:24.313 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.016733338, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.016433745, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.006979319, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 301, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.009977378, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000033147 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 502992, 2024-04-25T00:18:24.314 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 186160 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:24.315 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:24.316 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148704, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 29, 2024-04-25T00:18:24.317 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-04-25T00:18:24.318 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 928, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 307, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 459901, 2024-04-25T00:18:24.319 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 307, 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 307, 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 459901, 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:24.320 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:18:25.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:24 smithi110 ceph-mon[29187]: pgmap v842: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-04-25T00:18:25.118 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T00:18:25.118 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T00:18:25.118 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T00:18:25.118 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T00:18:25.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:24 smithi188 ceph-mon[33285]: pgmap v842: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-04-25T00:18:25.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:24 smithi173 ceph-mon[32402]: pgmap v842: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-04-25T00:18:25.359 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:27.143 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:26 smithi110 ceph-mon[29187]: pgmap v843: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 911 KiB/s wr, 110 op/s 2024-04-25T00:18:27.143 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:18:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 301, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 298, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 501904, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 183600, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.039971822, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.015696429, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.015027081, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.007150262, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 297, 2024-04-25T00:18:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.010762995, 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000036239 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 501904, 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 183600 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2, 2024-04-25T00:18:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 227, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 195344, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.014933222, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.011782244, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.001507728, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 225, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.009258075, 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000041147 2024-04-25T00:18:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 195344 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 4, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 2944, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 448, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-04-25T00:18:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.000659152, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000034872, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000623015, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000731975, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000243991 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148704, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 29, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 928, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-04-25T00:18:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-04-25T00:18:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-04-25T00:18:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-04-25T00:18:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 303, 2024-04-25T00:18:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 459381, 2024-04-25T00:18:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 303, 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 303, 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 459381, 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:27.155 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:18:27.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:26 smithi188 ceph-mon[33285]: pgmap v843: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 911 KiB/s wr, 110 op/s 2024-04-25T00:18:27.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:26 smithi173 ceph-mon[32402]: pgmap v843: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 911 KiB/s wr, 110 op/s 2024-04-25T00:18:27.877 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T00:18:27.877 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T00:18:27.878 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T00:18:27.878 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T00:18:28.115 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:28 smithi188 ceph-mon[33285]: pgmap v844: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 256 KiB/s wr, 47 op/s 2024-04-25T00:18:29.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:28 smithi110 ceph-mon[29187]: pgmap v844: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 256 KiB/s wr, 47 op/s 2024-04-25T00:18:29.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:28 smithi173 ceph-mon[32402]: pgmap v844: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 256 KiB/s wr, 47 op/s 2024-04-25T00:18:29.897 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T00:18:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 311, 2024-04-25T00:18:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 306, 2024-04-25T00:18:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 506480, 2024-04-25T00:18:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 187888, 2024-04-25T00:18:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-04-25T00:18:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.041097349, 2024-04-25T00:18:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.016591951, 2024-04-25T00:18:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.016333101, 2024-04-25T00:18:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.006626612, 2024-04-25T00:18:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 306, 2024-04-25T00:18:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.013240154, 2024-04-25T00:18:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000043268 2024-04-25T00:18:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 506480, 2024-04-25T00:18:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 187888 2024-04-25T00:18:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2, 2024-04-25T00:18:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 232, 2024-04-25T00:18:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:18:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 196224, 2024-04-25T00:18:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.015093774, 2024-04-25T00:18:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.011800234, 2024-04-25T00:18:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.001598980, 2024-04-25T00:18:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 230, 2024-04-25T00:18:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.008579198, 2024-04-25T00:18:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000037300 2024-04-25T00:18:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:18:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 196224 2024-04-25T00:18:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 0, 2024-04-25T00:18:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2, 2024-04-25T00:18:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:18:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 0, 2024-04-25T00:18:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-04-25T00:18:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-04-25T00:18:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.000480039, 2024-04-25T00:18:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:18:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000399629, 2024-04-25T00:18:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:18:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:18:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-04-25T00:18:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-04-25T00:18:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-04-25T00:18:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-04-25T00:18:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-04-25T00:18:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148744, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 30, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-04-25T00:18:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 928, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-04-25T00:18:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 309, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 460161, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 309, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 309, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 460161, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-04-25T00:18:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T00:18:29.922 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T00:18:30.675 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T00:18:30.675 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T00:18:30.675 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T00:18:30.675 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T00:18:31.019 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:18:31.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:30 smithi173 ceph-mon[32402]: pgmap v845: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-25T00:18:31.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:30 smithi188 ceph-mon[33285]: pgmap v845: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-25T00:18:31.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:30 smithi110 ceph-mon[29187]: pgmap v845: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-25T00:18:32.903 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-25T00:18:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 8006, 2024-04-25T00:18:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 9551, 2024-04-25T00:18:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 2314453, 2024-04-25T00:18:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 17524656, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 155, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.580397713, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.242426176, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.268176044, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.083066036, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 9552, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.735065963, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000076954 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 11032, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001801095, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000163 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2166826, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1087894, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 585805313, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 224428981, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 8, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 55.016504327, 2024-04-25T00:18:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 22.203199624, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 43.136293701, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.075610543, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1087896, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 13.476408897, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000012387 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3248831, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.275226877, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000084 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 489312 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 555970, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1098167, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 194618878, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 391399752, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 14, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 8, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 29.167612967, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 13.694358674, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 15.492393421, 2024-04-25T00:18:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.163708583, 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1098160, 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 15.231926257, 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000013870 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1530460, 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.173237785, 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000113 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 634016, 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 220352 2024-04-25T00:18:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 6597, 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 4.931006530, 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000747461 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-25T00:18:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 866, 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.033363631, 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000038526 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 6007, 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 4273, 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4273, 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 10.826046000, 2024-04-25T00:18:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002533593 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 1734, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 1167, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 3, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 883, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 0, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 867, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 16, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 883, 2024-04-25T00:18:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 945, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 863, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 863, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 6, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 7702, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 5698, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 1719, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 1, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 44213, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 31, 2024-04-25T00:18:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 937, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 27, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 804, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 1014, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 1467659605, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:18:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 32, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 541764, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 542201, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 1, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 546544, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 2675, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 59, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 2257 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 20, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 261, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 250, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 11, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:18:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 31, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 3, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 9261, 2024-04-25T00:18:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 9139, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 9139, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 122, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 169, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 83, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 82, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 82, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 1, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 1, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 1, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 26616788, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 26890133, 2024-04-25T00:18:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3272, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 24.939345866, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.007622049 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 878, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 2060, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 1182, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 65, 2024-04-25T00:18:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 77, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 12, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 883, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 2124, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 1241, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 863, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 2053, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 1190, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 74100, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 231804 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 7064, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 548, 2024-04-25T00:18:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 6007, 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 78, 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 1197, 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1214, 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 5.591374755, 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.004605745 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 452, 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.188695576, 2024-04-25T00:18:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000417468 2024-04-25T00:18:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 11, 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.019873608, 2024-04-25T00:18:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001806691 2024-04-25T00:18:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-25T00:18:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 918, 2024-04-25T00:18:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 3.128923731, 2024-04-25T00:18:32.924 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003408413 2024-04-25T00:18:32.924 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.924 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-25T00:18:32.924 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.924 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.924 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.925 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-25T00:18:32.925 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.925 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.925 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.925 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-25T00:18:32.925 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.927 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-25T00:18:32.927 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.927 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.927 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.927 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-25T00:18:32.927 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 37, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.043164720, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001166614 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.928 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1265, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.929055765, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000734431 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 81, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.135652027, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001674716 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 245, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.678673544, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002770096 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.929 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.008566887, 2024-04-25T00:18:32.930 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.004283443 2024-04-25T00:18:32.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.931 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-25T00:18:32.931 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.931 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.931 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.931 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-25T00:18:32.932 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.932 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.933 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.933 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-25T00:18:32.933 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-25T00:18:32.934 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.003370392, 2024-04-25T00:18:32.934 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001685196 2024-04-25T00:18:32.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.934 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 7, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.017968924, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002566989 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 39, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.080726071, 2024-04-25T00:18:32.936 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002069899 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 1, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 2, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 1, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 1, 2024-04-25T00:18:32.937 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 338, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 338, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 988, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 360, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 360, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:32.938 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:32.939 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 284191, 2024-04-25T00:18:32.940 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 109, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 7432, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 156, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-25T00:18:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 2859103, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 11543, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 7307, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 25693172, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 31, 2024-04-25T00:18:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 7307, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 7307, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 116.350354585, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.015923135 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 7307, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 12027 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 7307, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 105, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 7202, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-25T00:18:32.943 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 1096, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 1143, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 105, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 3377, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 348, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 3, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 802, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 2286, 2024-04-25T00:18:32.944 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 1, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-25T00:18:32.945 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 87, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-25T00:18:32.946 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 3, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 12, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 12, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 2687, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 179 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-25T00:18:32.947 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 21, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 9, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 280, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 250, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:32.948 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 2730798, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 536940063, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 2730798, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2730798, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 536940063, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.949 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 7307, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 25718422, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 7307, 2024-04-25T00:18:32.950 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 25718422, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:32.951 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 7307, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 7307, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 7307, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 7307, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:32.952 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 250, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 25250, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 250, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 105, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 25250, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x5618988a6500": { 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 9261, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 22695829, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 9261, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 3272, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 22695829, 2024-04-25T00:18:32.954 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:32.955 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:32.955 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:32.955 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:32.955 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.955 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:32.955 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-25T00:18:33.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:32 smithi173 ceph-mon[32402]: pgmap v846: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-25T00:18:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:32 smithi188 ceph-mon[33285]: pgmap v846: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-25T00:18:33.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:32 smithi110 ceph-mon[29187]: pgmap v846: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-25T00:18:33.718 INFO:tasks.check_counter:Daemon mds.b mds.exported=31 2024-04-25T00:18:33.718 INFO:tasks.check_counter:Daemon mds.b mds.imported=27 2024-04-25T00:18:33.718 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T00:18:33.719 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T00:18:33.967 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:18:35.182 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:34 smithi173 ceph-mon[32402]: pgmap v847: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T00:18:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:34 smithi188 ceph-mon[33285]: pgmap v847: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T00:18:35.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:34 smithi110 ceph-mon[29187]: pgmap v847: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T00:18:36.188 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-25T00:18:36.188 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:36.188 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 3192, 2024-04-25T00:18:36.188 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 3192, 2024-04-25T00:18:36.188 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 13489237, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 1014239, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.202017274, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.059339443, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.058151900, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.070050063, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3191, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.069048858, 2024-04-25T00:18:36.189 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000021638 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 366, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 596, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1854636, 2024-04-25T00:18:36.190 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 602340, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 5, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.035292237, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.013913070, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.008146968, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.009689041, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 594, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.016494506, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000027768 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.191 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 486208 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 326, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 312, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 565740, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 188942, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.037073967, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.014035593, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.016372700, 2024-04-25T00:18:36.192 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.006116387, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 311, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.008933166, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000028724 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 565472, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 188624 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-25T00:18:36.193 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 6013, 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.122299082, 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000020339 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.194 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.195 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 456, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 2, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 454, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 454, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 5, 2024-04-25T00:18:36.196 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-25T00:18:36.197 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:18:36.198 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2024-04-25T00:18:36.199 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:18:36.200 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 11379, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 0, 2024-04-25T00:18:36.201 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 15, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 18540894, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 18540320, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.202 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 11379 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 169, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 336, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 167, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 20, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 24, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 4, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 456, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 623, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 167, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-04-25T00:18:36.203 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 52012, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 215420 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.204 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-25T00:18:36.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.206 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-25T00:18:36.207 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.207 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.207 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.207 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.207 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-25T00:18:36.207 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.208 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.208 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.208 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-25T00:18:36.209 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.209 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.209 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.209 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-25T00:18:36.209 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.209 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.210 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.210 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-25T00:18:36.210 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.210 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.210 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.211 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-25T00:18:36.211 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.211 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.211 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.212 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-25T00:18:36.212 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.212 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.212 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.212 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.214 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.215 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.216 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 1, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 2, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 1, 2024-04-25T00:18:36.218 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 1, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 2064, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:36.219 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:36.220 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 190951, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 54, 2024-04-25T00:18:36.221 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 7432, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 156, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:36.222 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 707524, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 11956, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 3554, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 21, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 3, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 3554, 2024-04-25T00:18:36.223 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3554, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2.195951763, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000617881 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3554, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 3555 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 3554, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 3554, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 0, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-25T00:18:36.224 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 2135, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 1417, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 1, 2024-04-25T00:18:36.225 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-25T00:18:36.226 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 87, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-25T00:18:36.227 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 2, 2024-04-25T00:18:36.228 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 5, 2024-04-25T00:18:36.228 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 6, 2024-04-25T00:18:36.228 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 1, 2024-04-25T00:18:36.228 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-25T00:18:36.228 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 0, 2024-04-25T00:18:36.228 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 2, 2024-04-25T00:18:36.228 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 0 2024-04-25T00:18:36.229 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.229 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-25T00:18:36.229 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 0, 2024-04-25T00:18:36.229 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:18:36.229 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-25T00:18:36.229 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-25T00:18:36.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.230 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-25T00:18:36.230 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-25T00:18:36.230 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.230 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-25T00:18:36.230 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:36.231 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 3880, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 15540781, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 3880, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 3880, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 15540781, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.232 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-25T00:18:36.233 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:36.233 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-25T00:18:36.233 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:36.233 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-25T00:18:36.233 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-25T00:18:36.233 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 3554, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 14348365, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 3554, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 14348365, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.234 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 3554, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 3554, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 3554, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 3554, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.235 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-25T00:18:36.236 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x558c6adae6e0": { 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:36.237 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-25T00:18:36.238 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-25T00:18:36.238 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:36.238 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:36.238 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:36.238 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:36.238 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.238 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:36.238 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-25T00:18:37.124 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T00:18:37.124 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T00:18:37.124 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T00:18:37.124 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T00:18:37.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:36 smithi188 ceph-mon[33285]: pgmap v848: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T00:18:37.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:36 smithi110 ceph-mon[29187]: pgmap v848: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T00:18:37.349 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:36 smithi173 ceph-mon[32402]: pgmap v848: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T00:18:37.369 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:18:39.201 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:38 smithi173 ceph-mon[32402]: pgmap v849: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T00:18:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:38 smithi188 ceph-mon[33285]: pgmap v849: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T00:18:39.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:38 smithi110 ceph-mon[29187]: pgmap v849: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T00:18:39.330 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-25T00:18:39.330 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 7771, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 6217, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 2366005, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 14544085, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 9, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.441896298, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.200111145, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.247011739, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.038719057, 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:39.331 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 6215, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.147857684, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000023790 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 11057, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001888940, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000170 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 14973, 2024-04-25T00:18:39.332 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 15790, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 5413134, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 7623720, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 13, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.775205191, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.312994190, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.579685489, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.006679143, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 15790, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.408547416, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000025873 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.333 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 28909, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.004900238, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000169 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 582240, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 193856 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 536, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 765, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 144867, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 2444991, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 8, 2024-04-25T00:18:39.334 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.053028603, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.021280957, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.012662685, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.014439103, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 763, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.030171594, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000039543 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.335 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 574976 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3145, 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2.095308226, 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000666234 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.336 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 27, 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000552077, 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000020447 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 4313, 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 4201, 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4201, 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 3.612947232, 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000860020 2024-04-25T00:18:39.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 112, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 11, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 456, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 0, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 436, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 20, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 456, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 169, 2024-04-25T00:18:39.338 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 156, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 156, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 6, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 10300, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 8162, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 111, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 6, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 6315, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 4, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 166, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 7, 2024-04-25T00:18:39.339 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 275, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 1014, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 1467659605, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-25T00:18:39.340 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 8, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 419, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 741, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 10264, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 9156, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 2, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 462 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 6, 2024-04-25T00:18:39.341 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 154, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 2, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 152, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-25T00:18:39.342 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 4, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 11379, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 11379, 2024-04-25T00:18:39.343 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 15, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 15, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 18540894, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1436, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 3.133546958, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002182135 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.344 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 172, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 339, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 167, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 24, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 28, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 4, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 456, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 634, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 178, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 156, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 321, 2024-04-25T00:18:39.345 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 165, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 59472, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 223612 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 6517, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 5, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 4313, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 20, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 517, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 45, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.088890729, 2024-04-25T00:18:39.346 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001975349 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 10, 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.007234009, 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000723400 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 152, 2024-04-25T00:18:39.347 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.387587167, 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002549915 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3163, 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1.234827592, 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000390397 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.348 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.349 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-25T00:18:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-25T00:18:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 514, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.323713882, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000629793 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 11, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.008612422, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000782947 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.352 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 152, 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1.217395419, 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.008009180 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.354 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-25T00:18:39.355 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.356 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 154, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.344686012, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002238220 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.357 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 1, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 2, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 1, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 1, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 1747, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 1747, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 991, 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-25T00:18:39.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:39.359 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:39.360 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 280213, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 95, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:39.361 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 7432, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 156, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 802620, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 15280, 2024-04-25T00:18:39.362 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 1805, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 14401400, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 28, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 1805, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1805, 2024-04-25T00:18:39.363 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 4.425902779, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002452023 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1805, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2106 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 1805, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 2, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 1803, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2024-04-25T00:18:39.364 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 2, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 1438, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 71, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 20, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:18:39.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-25T00:18:39.366 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 87, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 2, 2024-04-25T00:18:39.367 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 10, 2024-04-25T00:18:39.368 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 10, 2024-04-25T00:18:39.368 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-04-25T00:18:39.368 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-25T00:18:39.368 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 412, 2024-04-25T00:18:39.368 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-04-25T00:18:39.368 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 133 2024-04-25T00:18:39.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.368 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 4, 2024-04-25T00:18:39.369 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 1, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 5, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 2, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 23276, 2024-04-25T00:18:39.370 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 5808035, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 23276, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 23276, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 5808035, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.371 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:39.372 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 1805, 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 14401602, 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1805, 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 14401602, 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.373 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 1805, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 1805, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1805, 2024-04-25T00:18:39.374 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1805, 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 2, 2024-04-25T00:18:39.375 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 202, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 2, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 202, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.376 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x555c36d54500": { 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 11379, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 14346590, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 11379, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:39.377 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1436, 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 14346590, 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:39.378 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-25T00:18:40.178 INFO:tasks.check_counter:Daemon mds.h mds.exported=4 2024-04-25T00:18:40.179 INFO:tasks.check_counter:Daemon mds.h mds.imported=7 2024-04-25T00:18:40.179 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T00:18:40.179 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T00:18:40.423 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:18:41.102 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:40 smithi173 ceph-mon[32402]: pgmap v850: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T00:18:41.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:40 smithi188 ceph-mon[33285]: pgmap v850: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T00:18:41.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:40 smithi110 ceph-mon[29187]: pgmap v850: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T00:18:42.300 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 4, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 4, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 2944, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 448, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 0, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.001310462, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.000038685, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.000465875, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:42.301 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000656719, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000218906 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2, 2024-04-25T00:18:42.302 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 235, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 196752, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.010369519, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.007550309, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.001099058, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 233, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.008186360, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000035134 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:42.303 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 196752 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 311, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 308, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 504624, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 190000, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 2, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 1, 2024-04-25T00:18:42.304 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.036070511, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.013803696, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.015195693, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.006154190, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 307, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.008941280, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000029124 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 504624, 2024-04-25T00:18:42.305 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 190000 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 0, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:42.306 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:42.307 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:42.308 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 148660, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 29, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-25T00:18:42.309 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 928, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 2, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 313, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 460681, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:42.310 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 313, 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 313, 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 460681, 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-25T00:18:42.311 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-25T00:18:43.029 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:42 smithi173 ceph-mon[32402]: pgmap v851: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T00:18:43.058 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T00:18:43.058 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T00:18:43.058 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T00:18:43.058 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T00:18:43.155 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:42 smithi188 ceph-mon[33285]: pgmap v851: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T00:18:43.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:42 smithi110 ceph-mon[29187]: pgmap v851: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T00:18:43.394 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:18:45.193 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:44 smithi188 ceph-mon[33285]: pgmap v852: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:45.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:44 smithi110 ceph-mon[29187]: pgmap v852: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:45.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:44 smithi173 ceph-mon[32402]: pgmap v852: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:45.790 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2068, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2068, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 9008197, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 656810, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.153123470, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.042578878, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.043872294, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.053380832, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2067, 2024-04-25T00:18:45.791 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.061079617, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000029549 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1079, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1310, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 8666930, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 919496, 2024-04-25T00:18:45.792 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.079813891, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.028003433, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.022977572, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.022341558, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1308, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.033923808, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000025935 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.793 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 576944 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1004, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 990, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 6664767, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 404711, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:45.794 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 3, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.088534477, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.028727819, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.031044351, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.023977032, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 989, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.027979464, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000028290 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.795 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 565904, 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 189424 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 6478, 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.168550681, 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000026018 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-04-25T00:18:45.796 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-04-25T00:18:45.797 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-04-25T00:18:45.798 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 1179, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 778, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 401, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 75, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 6, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-04-25T00:18:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-04-25T00:18:45.800 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 1014, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 1467659605, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:18:45.801 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:18:45.803 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-04-25T00:18:45.804 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 9266, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 75, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 26616788, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 26901274, 2024-04-25T00:18:45.805 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 26899363, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 9266 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 881, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 2060, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 1179, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 67, 2024-04-25T00:18:45.806 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 77, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 10, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 1179, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 2118, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 939, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 57928, 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215420 2024-04-25T00:18:45.807 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.808 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-04-25T00:18:45.808 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-04-25T00:18:45.808 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-04-25T00:18:45.808 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-04-25T00:18:45.808 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-04-25T00:18:45.808 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-04-25T00:18:45.809 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-04-25T00:18:45.809 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.809 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.809 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.809 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.809 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-04-25T00:18:45.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.810 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.810 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.810 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.810 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-04-25T00:18:45.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.811 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.811 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.811 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.811 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-04-25T00:18:45.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.812 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.812 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.812 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-04-25T00:18:45.812 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.813 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.813 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.813 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.813 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-04-25T00:18:45.813 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.814 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.814 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.814 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-04-25T00:18:45.814 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.815 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.815 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.815 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-04-25T00:18:45.815 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.815 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.816 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.816 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-04-25T00:18:45.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.816 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.816 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.817 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-04-25T00:18:45.817 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.817 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.817 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.817 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.818 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-04-25T00:18:45.818 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.818 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.818 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.818 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.819 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-04-25T00:18:45.820 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.820 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.820 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.820 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.820 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-04-25T00:18:45.820 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.821 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.821 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.821 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.821 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-04-25T00:18:45.821 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.822 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.822 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.822 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.822 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-04-25T00:18:45.822 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.822 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.823 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.823 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.823 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-04-25T00:18:45.823 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.823 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.823 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.824 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.824 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-04-25T00:18:45.824 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.824 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.824 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.825 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-04-25T00:18:45.825 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.825 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.825 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.825 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.826 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-04-25T00:18:45.826 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.826 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.826 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.826 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.827 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-04-25T00:18:45.827 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.827 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.827 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.827 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.827 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-04-25T00:18:45.828 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.828 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.828 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.828 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.828 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-04-25T00:18:45.828 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.829 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.829 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.829 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.829 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-04-25T00:18:45.829 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.829 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.830 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.830 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-04-25T00:18:45.830 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.830 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.830 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.830 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.831 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-04-25T00:18:45.832 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 2, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 1, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-04-25T00:18:45.833 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 2083, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:45.834 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:45.835 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:45.838 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:45.838 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:45.838 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:45.838 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:45.838 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:45.838 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:45.838 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 199055, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 58, 2024-04-25T00:18:45.839 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 7432, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 156, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-04-25T00:18:45.840 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 2717511, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 6338, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 3818, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 26, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 4, 2024-04-25T00:18:45.841 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 3818, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3818, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.867999166, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000489261 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3818, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 3819 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 3818, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 3818, 2024-04-25T00:18:45.842 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 2211, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 1604, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-04-25T00:18:45.843 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 2, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-04-25T00:18:45.844 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-04-25T00:18:45.845 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 87, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 6, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 6, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.846 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-04-25T00:18:45.847 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 4147, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 23946891, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 4147, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:45.848 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:45.849 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 4147, 2024-04-25T00:18:45.849 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 23946891, 2024-04-25T00:18:45.849 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:45.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.849 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.849 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3818, 2024-04-25T00:18:45.850 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 22711528, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3818, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 22711528, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:45.851 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3818, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 3818, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3818, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 3818, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:45.852 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.853 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x5640faf046e0": { 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:45.855 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:45.855 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:45.855 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.855 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:45.855 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-25T00:18:46.884 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T00:18:46.884 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T00:18:46.885 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T00:18:46.885 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T00:18:47.142 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:18:47.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:46 smithi188 ceph-mon[33285]: pgmap v853: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:47.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:46 smithi110 ceph-mon[29187]: pgmap v853: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:47.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:46 smithi173 ceph-mon[32402]: pgmap v853: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:49.167 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:48 smithi188 ceph-mon[33285]: pgmap v854: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T00:18:49.168 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-25T00:18:49.168 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:49.168 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1286, 2024-04-25T00:18:49.168 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1288, 2024-04-25T00:18:49.168 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 1747342, 2024-04-25T00:18:49.168 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 409012, 2024-04-25T00:18:49.168 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 7, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.085418999, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.025464535, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.023586496, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.027874629, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1286, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.036569280, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000028436 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.169 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1377, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1361, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 826324, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 523307, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 3, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.108869909, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.033078485, 2024-04-25T00:18:49.170 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.032585508, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.036574706, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1361, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.036990905, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000027179 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 569232, 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 190992 2024-04-25T00:18:49.171 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1121, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1353, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 376480, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 845952, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 3, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.102582750, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.035678799, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.023970144, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.033426727, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1351, 2024-04-25T00:18:49.172 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.044516824, 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000032951 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 489792 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-04-25T00:18:49.173 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5749, 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.139101732, 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000024195 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.174 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-04-25T00:18:49.175 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 45, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 16, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 29, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 22, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 3, 2024-04-25T00:18:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-04-25T00:18:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-04-25T00:18:49.178 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-04-25T00:18:49.179 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-04-25T00:18:49.180 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 734, 2024-04-25T00:18:49.181 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 5606683, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 5606109, 2024-04-25T00:18:49.182 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 734 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 43, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 721, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 678, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 11, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 42, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 31, 2024-04-25T00:18:49.183 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 45, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 726, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 681, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 53644, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215420 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-04-25T00:18:49.184 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.185 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.186 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.187 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.188 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-04-25T00:18:49.189 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.189 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.189 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.189 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.189 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-04-25T00:18:49.189 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.189 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-04-25T00:18:49.190 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.191 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.192 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.193 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.194 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:18:49.195 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 2, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 1, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 2085, 2024-04-25T00:18:49.196 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:49.197 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:49.198 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 190167, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 52, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-04-25T00:18:49.199 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 7432, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 156, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 135693, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 329, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-04-25T00:18:49.200 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 3451, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 21, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 3, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 3451, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3451, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 2.000999224, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000579831 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.201 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3451, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 3452 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 3451, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 3451, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 2163, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 1286, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-04-25T00:18:49.202 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 1, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-04-25T00:18:49.203 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-04-25T00:18:49.204 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 87, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 5, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 6, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 1, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-04-25T00:18:49.205 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:18:49.206 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 3780, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 2590158, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 3780, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:49.207 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3780, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 2590158, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:49.208 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3451, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 1412384, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3451, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1412384, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:49.209 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 3451, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 3451, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 3451, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 3451, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:18:49.210 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.211 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x564c7dcfc6e0": { 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:49.212 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:49.213 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:49.213 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:49.213 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.213 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:49.213 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-25T00:18:49.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:48 smithi110 ceph-mon[29187]: pgmap v854: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T00:18:49.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:48 smithi173 ceph-mon[32402]: pgmap v854: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T00:18:50.110 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T00:18:50.110 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T00:18:50.110 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T00:18:50.110 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T00:18:50.350 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:18:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:50 smithi188 ceph-mon[33285]: pgmap v855: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:51.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:50 smithi110 ceph-mon[29187]: pgmap v855: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:51.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:50 smithi173 ceph-mon[32402]: pgmap v855: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T00:18:52.096 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-25T00:18:52.096 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1180260, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1721192, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 361244593, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 752662083, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 15, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 53.442003049, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 23.352846884, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 34.255451557, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.010583350, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1721190, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 26.076327195, 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000015150 2024-04-25T00:18:52.097 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2835141, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.366271368, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000129 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 1440 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 202, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 431, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 121055, 2024-04-25T00:18:52.098 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 703878, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 10, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.025017001, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.012022378, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.008678567, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.003014868, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 432, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.029709377, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000068771 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 168, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000025949, 2024-04-25T00:18:52.099 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000154 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 579008 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1085627, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2164757, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 216961619, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 585303990, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 6, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 58.191755482, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 25.606960786, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 34.305553857, 2024-04-25T00:18:52.100 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.009723041, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2164754, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 30.140594397, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000013923 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3230536, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.432158861, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000133 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 575216, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 194240 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.101 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 836, 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.218354246, 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000261189 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-25T00:18:52.102 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 17, 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000324038, 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000019061 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 616682, 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 616625, 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 616625, 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 477.209599528, 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000773905 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.103 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 57, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 11, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 58, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 4, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 34, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 20, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 51, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 684, 2024-04-25T00:18:52.104 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 34, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 34, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 4, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 1232927, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 1232621, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 56, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 14, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 7551, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 24, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 672, 2024-04-25T00:18:52.105 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 25, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 696, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 1014, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 1467659605, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:18:52.106 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 34, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 562019, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 540455, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 563147, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 243, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 2, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 12 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-04-25T00:18:52.107 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 14, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 2, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 2, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-04-25T00:18:52.108 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 24, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 734, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-04-25T00:18:52.109 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 734, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 2, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 2, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 2, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 4194304, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 5606683, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-04-25T00:18:52.110 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 314, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.830187077, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002643907 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 62, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 731, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 669, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 45, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 53, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 8, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 58, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 749, 2024-04-25T00:18:52.111 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 691, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 34, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 703, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 669, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 60144, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 223612 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 1156949, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 95, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 616682, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 40, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 231, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 7, 2024-04-25T00:18:52.112 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.018965298, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002709328 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 615766, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 476.527406437, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000773877 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.113 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 439, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.409023357, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000931716 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.114 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.115 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.116 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.116 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-04-25T00:18:52.116 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.116 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.116 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.116 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.117 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 408, 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.248598050, 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000609308 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3, 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.001378425, 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000459475 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.118 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-04-25T00:18:52.119 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.120 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.121 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.122 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.004227961, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002113980 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.123 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 2, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 1, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 58362, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 58362, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 1004, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:52.124 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:52.125 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-04-25T00:18:52.126 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 266295, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 103, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:52.127 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 7432, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 156, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 249437, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 1264, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-04-25T00:18:52.128 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 504, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 1447026, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 28, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 504, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 504, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.270999187, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002521823 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 504, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 623 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 504, 2024-04-25T00:18:52.129 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 1, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 503, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 10, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 1, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 315, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 60, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 14, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-04-25T00:18:52.130 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 20, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-04-25T00:18:52.131 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 87, 2024-04-25T00:18:52.132 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 9, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 14, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 5, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 164, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 0, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 39 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-04-25T00:18:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 2, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 2, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 2, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 2, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:52.134 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 2266085, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 374358331, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 2266085, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 2266085, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 374358331, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.135 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 504, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 1447228, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 504, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1447228, 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:52.136 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 504, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 504, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 504, 2024-04-25T00:18:52.137 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 504, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 2, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 202, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:52.138 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 2, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 1, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 202, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x5569b88a6500": { 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 734, 2024-04-25T00:18:52.139 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1412379, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 734, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 314, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1412379, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:52.140 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-25T00:18:52.923 INFO:tasks.check_counter:Daemon mds.i mds.exported=24 2024-04-25T00:18:52.923 INFO:tasks.check_counter:Daemon mds.i mds.imported=25 2024-04-25T00:18:52.923 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T00:18:52.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 787fa75a-0295-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T00:18:52.945 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:52 smithi188 ceph-mon[33285]: pgmap v856: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-25T00:18:53.161 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:18:53.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:52 smithi110 ceph-mon[29187]: pgmap v856: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-25T00:18:53.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:52 smithi173 ceph-mon[32402]: pgmap v856: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-25T00:18:54.839 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-25T00:18:54.839 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:18:54.839 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 0, 2024-04-25T00:18:54.839 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 0, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 0, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.001081615, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.000308499, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.840 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 6, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 240, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 3264, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 197728, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-04-25T00:18:54.841 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.011431111, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.008206368, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.001377539, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 239, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.011377817, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000047605 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T00:18:54.842 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 197728 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 315, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 312, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 505712, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 192560, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 2, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 1, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.038809927, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.014796640, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.016937328, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.006459128, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 311, 2024-04-25T00:18:54.843 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.010632649, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000034188 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 505712, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 192560 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 0, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-04-25T00:18:54.844 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:18:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:18:54.846 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 148704, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 29, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:18:54.847 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 928, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 2, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:18:54.848 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 317, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 461201, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 317, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 317, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 461201, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-25T00:18:54.849 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:54.850 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-25T00:18:54.850 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-25T00:18:55.210 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:54 smithi188 ceph-mon[33285]: pgmap v857: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:18:55.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:54 smithi110 ceph-mon[29187]: pgmap v857: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:18:55.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:54 smithi173 ceph-mon[32402]: pgmap v857: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:18:55.539 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T00:18:55.539 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T00:18:55.540 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T00:18:55.550 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T00:18:55.550 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T00:18:55.550 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:18:55.571 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T00:18:55.571 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:18:55.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 -- 172.21.15.110:0/215537576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8101870 msgr2=0x7f7ee8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 --2- 172.21.15.110:0/215537576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8101870 0x7f7ee8101cf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009860 tx=0x7f7edc02a340 comp rx=0 tx=0).stop 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 -- 172.21.15.110:0/215537576 shutdown_connections 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 --2- 172.21.15.110:0/215537576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8102230 0x7f7ee8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 --2- 172.21.15.110:0/215537576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8101870 0x7f7ee8101cf0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 --2- 172.21.15.110:0/215537576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee8100680 0x7f7ee8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 -- 172.21.15.110:0/215537576 >> 172.21.15.110:0/215537576 conn(0x7f7ee80fbbf0 msgr2=0x7f7ee80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 -- 172.21.15.110:0/215537576 shutdown_connections 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 -- 172.21.15.110:0/215537576 wait complete. 2024-04-25T00:18:55.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.772+0000 7f7eed63f640 1 Processor -- start 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7eed63f640 1 -- start start 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7eed63f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8100680 0x7f7ee810d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7eed63f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee8101870 0x7f7ee810be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7eed63f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8102230 0x7f7ee810c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7eed63f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8068da0 con 0x7f7ee8100680 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7eed63f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8068c20 con 0x7f7ee8102230 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7eed63f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8068f20 con 0x7f7ee8101870 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8100680 0x7f7ee810d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8100680 0x7f7ee810d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59736/0 (socket says 172.21.15.110:59736) 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 -- 172.21.15.110:0/2104838825 learned_addr learned my addr 172.21.15.110:0/2104838825 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee67fc640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee8101870 0x7f7ee810be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:55.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee77fe640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8102230 0x7f7ee810c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 -- 172.21.15.110:0/2104838825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee8101870 msgr2=0x7f7ee810be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee8101870 0x7f7ee810be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 -- 172.21.15.110:0/2104838825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8102230 msgr2=0x7f7ee810c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8102230 0x7f7ee810c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 -- 172.21.15.110:0/2104838825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee810cc50 con 0x7f7ee8100680 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee67fc640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee8101870 0x7f7ee810be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.773+0000 7f7ee6ffd640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8100680 0x7f7ee810d840 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000e9a0 tx=0x7f7ee000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.774+0000 7f7ecffff640 1 -- 172.21.15.110:0/2104838825 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee000cd80 con 0x7f7ee8100680 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.774+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee81a1ac0 con 0x7f7ee8100680 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.774+0000 7f7ecffff640 1 -- 172.21.15.110:0/2104838825 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee00108e0 con 0x7f7ee8100680 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.774+0000 7f7ecffff640 1 -- 172.21.15.110:0/2104838825 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00188a0 con 0x7f7ee8100680 2024-04-25T00:18:55.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.774+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee81a2030 con 0x7f7ee8100680 2024-04-25T00:18:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.775+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee81078d0 con 0x7f7ee8100680 2024-04-25T00:18:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.776+0000 7f7ecffff640 1 -- 172.21.15.110:0/2104838825 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7ee0010a90 con 0x7f7ee8100680 2024-04-25T00:18:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.776+0000 7f7ecffff640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7ed00a2b90 0x7f7ed00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.776+0000 7f7ecffff640 1 -- 172.21.15.110:0/2104838825 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f7ee00cfa30 con 0x7f7ee8100680 2024-04-25T00:18:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.776+0000 7f7ee67fc640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7ed00a2b90 0x7f7ed00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:55.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.777+0000 7f7ee67fc640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7ed00a2b90 0x7f7ed00a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7edc02a930 tx=0x7f7edc030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:55.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.784+0000 7f7ecffff640 1 -- 172.21.15.110:0/2104838825 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ee0026070 con 0x7f7ee8100680 2024-04-25T00:18:55.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.968+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ee802d3d0 con 0x7f7ee8100680 2024-04-25T00:18:55.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.971+0000 7f7ecffff640 1 -- 172.21.15.110:0/2104838825 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7f7ee009c9c0 con 0x7f7ee8100680 2024-04-25T00:18:55.972 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:55.972 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7ed00a2b90 msgr2=0x7f7ed00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7ed00a2b90 0x7f7ed00a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7edc02a930 tx=0x7f7edc030450 comp rx=0 tx=0).stop 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8100680 msgr2=0x7f7ee810d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8100680 0x7f7ee810d840 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000e9a0 tx=0x7f7ee000ee60 comp rx=0 tx=0).stop 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 shutdown_connections 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f7ed00a2b90 0x7f7ed00a5050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8102230 0x7f7ee810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee8101870 0x7f7ee810be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 --2- 172.21.15.110:0/2104838825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ee8100680 0x7f7ee810d840 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:55.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 >> 172.21.15.110:0/2104838825 conn(0x7f7ee80fbbf0 msgr2=0x7f7ee8104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:55.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 shutdown_connections 2024-04-25T00:18:55.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:55.973+0000 7f7eed63f640 1 -- 172.21.15.110:0/2104838825 wait complete. 2024-04-25T00:18:55.985 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:01.224042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:02.090769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:55.985 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:18:56.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 -- 172.21.15.110:0/766141883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4101870 msgr2=0x7fade4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 --2- 172.21.15.110:0/766141883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4101870 0x7fade4101cf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009900 tx=0x7fadd802a410 comp rx=0 tx=0).stop 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 -- 172.21.15.110:0/766141883 shutdown_connections 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 --2- 172.21.15.110:0/766141883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fade4102230 0x7fade4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 --2- 172.21.15.110:0/766141883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4101870 0x7fade4101cf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 --2- 172.21.15.110:0/766141883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade4100680 0x7fade4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 -- 172.21.15.110:0/766141883 >> 172.21.15.110:0/766141883 conn(0x7fade40fbbf0 msgr2=0x7fade40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 -- 172.21.15.110:0/766141883 shutdown_connections 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.087+0000 7fade8d96640 1 -- 172.21.15.110:0/766141883 wait complete. 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade8d96640 1 Processor -- start 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade8d96640 1 -- start start 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade8d96640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade4100680 0x7fade4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade8d96640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fade4101870 0x7fade419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade8d96640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4102230 0x7fade419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade8d96640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fade410c190 con 0x7fade4102230 2024-04-25T00:18:56.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade8d96640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fade410c010 con 0x7fade4100680 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade8d96640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fade410c310 con 0x7fade4101870 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4102230 0x7fade419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4102230 0x7fade419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59758/0 (socket says 172.21.15.110:59758) 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.088+0000 7fade2d9d640 1 -- 172.21.15.110:0/2184516191 learned_addr learned my addr 172.21.15.110:0/2184516191 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade1d9b640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fade4101870 0x7fade419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade259c640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade4100680 0x7fade4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade2d9d640 1 -- 172.21.15.110:0/2184516191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fade4101870 msgr2=0x7fade419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade2d9d640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fade4101870 0x7fade419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade2d9d640 1 -- 172.21.15.110:0/2184516191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade4100680 msgr2=0x7fade4069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade2d9d640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade4100680 0x7fade4069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade2d9d640 1 -- 172.21.15.110:0/2184516191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade419f590 con 0x7fade4102230 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade259c640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade4100680 0x7fade4069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:56.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade2d9d640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4102230 0x7fade419ecd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faddc00eaa0 tx=0x7faddc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:56.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fadd37fe640 1 -- 172.21.15.110:0/2184516191 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc00cdb0 con 0x7fade4102230 2024-04-25T00:18:56.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade419f880 con 0x7fade4102230 2024-04-25T00:18:56.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fadd37fe640 1 -- 172.21.15.110:0/2184516191 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faddc005020 con 0x7fade4102230 2024-04-25T00:18:56.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fadd37fe640 1 -- 172.21.15.110:0/2184516191 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc0116d0 con 0x7fade4102230 2024-04-25T00:18:56.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.089+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade419fd50 con 0x7fade4102230 2024-04-25T00:18:56.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.090+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade4107a20 con 0x7fade4102230 2024-04-25T00:18:56.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.091+0000 7fadd37fe640 1 -- 172.21.15.110:0/2184516191 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7faddc0051d0 con 0x7fade4102230 2024-04-25T00:18:56.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.092+0000 7fadd37fe640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fadcc0a2b90 0x7fadcc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.092+0000 7fade259c640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fadcc0a2b90 0x7fadcc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.092+0000 7fadd37fe640 1 -- 172.21.15.110:0/2184516191 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7faddc0cf200 con 0x7fade4102230 2024-04-25T00:18:56.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.092+0000 7fade259c640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fadcc0a2b90 0x7fadcc0a5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fade41016d0 tx=0x7fadd4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:56.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.100+0000 7fadd37fe640 1 -- 172.21.15.110:0/2184516191 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faddc016030 con 0x7fade4102230 2024-04-25T00:18:56.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.279+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fade402d300 con 0x7fade4102230 2024-04-25T00:18:56.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.280+0000 7fadd37fe640 1 -- 172.21.15.110:0/2184516191 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7faddc09c190 con 0x7fade4102230 2024-04-25T00:18:56.280 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T00:18:56.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fadcc0a2b90 msgr2=0x7fadcc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fadcc0a2b90 0x7fadcc0a5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fade41016d0 tx=0x7fadd4005db0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4102230 msgr2=0x7fade419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4102230 0x7fade419ecd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faddc00eaa0 tx=0x7faddc00ef60 comp rx=0 tx=0).stop 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 shutdown_connections 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fadcc0a2b90 0x7fadcc0a5050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade4102230 0x7fade419ecd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fade4101870 0x7fade419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 --2- 172.21.15.110:0/2184516191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade4100680 0x7fade4069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 >> 172.21.15.110:0/2184516191 conn(0x7fade40fbbf0 msgr2=0x7fade4104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 shutdown_connections 2024-04-25T00:18:56.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.282+0000 7fade8d96640 1 -- 172.21.15.110:0/2184516191 wait complete. 2024-04-25T00:18:56.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:56 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2104838825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:56.293 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:18:56.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:56 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2104838825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:56.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 -- 172.21.15.110:0/188134434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c102250 msgr2=0x7ffb5c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/188134434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c102250 0x7ffb5c108ae0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009ca0 tx=0x7ffb5402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 -- 172.21.15.110:0/188134434 shutdown_connections 2024-04-25T00:18:56.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/188134434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c102250 0x7ffb5c108ae0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/188134434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5c101890 0x7ffb5c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/188134434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb5c1006a0 0x7ffb5c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 -- 172.21.15.110:0/188134434 >> 172.21.15.110:0/188134434 conn(0x7ffb5c0fbc30 msgr2=0x7ffb5c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 -- 172.21.15.110:0/188134434 shutdown_connections 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 -- 172.21.15.110:0/188134434 wait complete. 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 Processor -- start 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.404+0000 7ffb62cd4640 1 -- start start 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb62cd4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb5c1006a0 0x7ffb5c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb62cd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c101890 0x7ffb5c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb62cd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5c102250 0x7ffb5c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb62cd4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c10c1d0 con 0x7ffb5c101890 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb62cd4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c10c050 con 0x7ffb5c1006a0 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb62cd4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c10c350 con 0x7ffb5c102250 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c101890 0x7ffb5c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c101890 0x7ffb5c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59762/0 (socket says 172.21.15.110:59762) 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 -- 172.21.15.110:0/3856195641 learned_addr learned my addr 172.21.15.110:0/3856195641 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:56.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb60a70640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb5c1006a0 0x7ffb5c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb61271640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5c102250 0x7ffb5c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 -- 172.21.15.110:0/3856195641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5c102250 msgr2=0x7ffb5c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5c102250 0x7ffb5c19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 -- 172.21.15.110:0/3856195641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb5c1006a0 msgr2=0x7ffb5c06a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb5c1006a0 0x7ffb5c06a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 -- 172.21.15.110:0/3856195641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c19f5d0 con 0x7ffb5c101890 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb53fff640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c101890 0x7ffb5c19c6d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00c970 tx=0x7ffb4c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb51ffb640 1 -- 172.21.15.110:0/3856195641 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c007bb0 con 0x7ffb5c101890 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb51ffb640 1 -- 172.21.15.110:0/3856195641 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4c007d50 con 0x7ffb5c101890 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5c19f8c0 con 0x7ffb5c101890 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb51ffb640 1 -- 172.21.15.110:0/3856195641 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00f450 con 0x7ffb5c101890 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.405+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5c19fdc0 con 0x7ffb5c101890 2024-04-25T00:18:56.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.406+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb5c107a40 con 0x7ffb5c101890 2024-04-25T00:18:56.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.407+0000 7ffb51ffb640 1 -- 172.21.15.110:0/3856195641 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffb4c004f20 con 0x7ffb5c101890 2024-04-25T00:18:56.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.408+0000 7ffb51ffb640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.408+0000 7ffb60a70640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.408+0000 7ffb51ffb640 1 -- 172.21.15.110:0/3856195641 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ffb4c0cedb0 con 0x7ffb5c101890 2024-04-25T00:18:56.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.408+0000 7ffb60a70640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ffb440a2b90 0x7ffb440a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005d00 tx=0x7ffb58005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:56.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.412+0000 7ffb51ffb640 1 -- 172.21.15.110:0/3856195641 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb4c014030 con 0x7ffb5c101890 2024-04-25T00:18:56.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:56 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2104838825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:56.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.587+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffb5c02d170 con 0x7ffb5c101890 2024-04-25T00:18:56.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.587+0000 7ffb51ffb640 1 -- 172.21.15.110:0/3856195641 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7ffb4c097030 con 0x7ffb5c101890 2024-04-25T00:18:56.588 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T00:18:56.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ffb440a2b90 msgr2=0x7ffb440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ffb440a2b90 0x7ffb440a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005d00 tx=0x7ffb58005c50 comp rx=0 tx=0).stop 2024-04-25T00:18:56.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c101890 msgr2=0x7ffb5c19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c101890 0x7ffb5c19c6d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00c970 tx=0x7ffb4c00ce30 comp rx=0 tx=0).stop 2024-04-25T00:18:56.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 shutdown_connections 2024-04-25T00:18:56.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ffb440a2b90 0x7ffb440a5050 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb5c102250 0x7ffb5c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffb5c101890 0x7ffb5c19c6d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 --2- 172.21.15.110:0/3856195641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb5c1006a0 0x7ffb5c06a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 >> 172.21.15.110:0/3856195641 conn(0x7ffb5c0fbc30 msgr2=0x7ffb5c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:56.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 shutdown_connections 2024-04-25T00:18:56.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.589+0000 7ffb62cd4640 1 -- 172.21.15.110:0/3856195641 wait complete. 2024-04-25T00:18:56.600 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:18:56.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 -- 172.21.15.110:0/3954938070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 msgr2=0x7f64701020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 --2- 172.21.15.110:0/3954938070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 0x7f64701020d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6460009900 tx=0x7f646002a410 comp rx=0 tx=0).stop 2024-04-25T00:18:56.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 -- 172.21.15.110:0/3954938070 shutdown_connections 2024-04-25T00:18:56.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 --2- 172.21.15.110:0/3954938070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64701012c0 0x7f6470108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 --2- 172.21.15.110:0/3954938070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6470100900 0x7f6470100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 --2- 172.21.15.110:0/3954938070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 0x7f64701020d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 -- 172.21.15.110:0/3954938070 >> 172.21.15.110:0/3954938070 conn(0x7f64700fb370 msgr2=0x7f64700fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 -- 172.21.15.110:0/3954938070 shutdown_connections 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.704+0000 7f647586b640 1 -- 172.21.15.110:0/3954938070 wait complete. 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f647586b640 1 Processor -- start 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f647586b640 1 -- start start 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f647586b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6470100900 0x7f6470100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f647586b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64701012c0 0x7f647019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f647586b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 0x7f647019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f647586b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f647010c1c0 con 0x7f6470101cb0 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f647586b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f647010c040 con 0x7f6470100900 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f647586b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f647010c340 con 0x7f64701012c0 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 0x7f647019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f646f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 0x7f647019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46892/0 (socket says 172.21.15.110:46892) 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f646f7fe640 1 -- 172.21.15.110:0/197170400 learned_addr learned my addr 172.21.15.110:0/197170400 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f646effd640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6470100900 0x7f6470100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.705+0000 7f646e7fc640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64701012c0 0x7f647019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f646f7fe640 1 -- 172.21.15.110:0/197170400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64701012c0 msgr2=0x7f647019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f646f7fe640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64701012c0 0x7f647019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f646f7fe640 1 -- 172.21.15.110:0/197170400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6470100900 msgr2=0x7f6470100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f646f7fe640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6470100900 0x7f6470100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f646f7fe640 1 -- 172.21.15.110:0/197170400 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647019f650 con 0x7f6470101cb0 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f646effd640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6470100900 0x7f6470100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f646f7fe640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 0x7f647019ed90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f646800eaa0 tx=0x7f646800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f6474869640 1 -- 172.21.15.110:0/197170400 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646800cdb0 con 0x7f6470101cb0 2024-04-25T00:18:56.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647019f940 con 0x7f6470101cb0 2024-04-25T00:18:56.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f6474869640 1 -- 172.21.15.110:0/197170400 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6468005020 con 0x7f6470101cb0 2024-04-25T00:18:56.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f6474869640 1 -- 172.21.15.110:0/197170400 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64680116d0 con 0x7f6470101cb0 2024-04-25T00:18:56.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.706+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647019fe40 con 0x7f6470101cb0 2024-04-25T00:18:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.707+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6470107ba0 con 0x7f6470101cb0 2024-04-25T00:18:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.708+0000 7f6474869640 1 -- 172.21.15.110:0/197170400 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f64680051d0 con 0x7f6470101cb0 2024-04-25T00:18:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.709+0000 7f6474869640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64580a2b90 0x7f64580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.709+0000 7f646effd640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64580a2b90 0x7f64580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.709+0000 7f6474869640 1 -- 172.21.15.110:0/197170400 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f64680cf110 con 0x7f6470101cb0 2024-04-25T00:18:56.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.710+0000 7f646effd640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64580a2b90 0x7f64580a5050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6460004770 tx=0x7f6460000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:56.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.718+0000 7f6474869640 1 -- 172.21.15.110:0/197170400 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6468016030 con 0x7f6470101cb0 2024-04-25T00:18:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.906+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6470062340 con 0x7f6470101cb0 2024-04-25T00:18:56.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.906+0000 7f6474869640 1 -- 172.21.15.110:0/197170400 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f646809ba50 con 0x7f6470101cb0 2024-04-25T00:18:56.907 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 64 2024-04-25T00:18:56.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64580a2b90 msgr2=0x7f64580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64580a2b90 0x7f64580a5050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6460004770 tx=0x7f6460000bc0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 msgr2=0x7f647019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 0x7f647019ed90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f646800eaa0 tx=0x7f646800ef60 comp rx=0 tx=0).stop 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 shutdown_connections 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64580a2b90 0x7f64580a5050 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6470101cb0 0x7f647019ed90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64701012c0 0x7f647019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 --2- 172.21.15.110:0/197170400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6470100900 0x7f6470100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.908+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 >> 172.21.15.110:0/197170400 conn(0x7f64700fb370 msgr2=0x7f6470104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:56.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.909+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 shutdown_connections 2024-04-25T00:18:56.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:56.909+0000 7f647586b640 1 -- 172.21.15.110:0/197170400 wait complete. 2024-04-25T00:18:56.921 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:18:57.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 -- 172.21.15.110:0/568802278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440100680 msgr2=0x7f6440100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 --2- 172.21.15.110:0/568802278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440100680 0x7f6440100aa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6438009900 tx=0x7f643802a410 comp rx=0 tx=0).stop 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 -- 172.21.15.110:0/568802278 shutdown_connections 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 --2- 172.21.15.110:0/568802278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102230 0x7f6440108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 --2- 172.21.15.110:0/568802278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6440101870 0x7f6440101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 --2- 172.21.15.110:0/568802278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440100680 0x7f6440100aa0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 -- 172.21.15.110:0/568802278 >> 172.21.15.110:0/568802278 conn(0x7f64400fbbf0 msgr2=0x7f64400fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 -- 172.21.15.110:0/568802278 shutdown_connections 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.023+0000 7f64456a5640 1 -- 172.21.15.110:0/568802278 wait complete. 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.024+0000 7f64456a5640 1 Processor -- start 2024-04-25T00:18:57.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.024+0000 7f64456a5640 1 -- start start 2024-04-25T00:18:57.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.024+0000 7f64456a5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6440100680 0x7f6440196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.024+0000 7f64456a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440101870 0x7f6440196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.024+0000 7f64456a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102230 0x7f644019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.024+0000 7f64456a5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c1b0 con 0x7f6440101870 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.024+0000 7f64456a5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c030 con 0x7f6440100680 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.024+0000 7f64456a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c330 con 0x7f6440102230 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440101870 0x7f6440196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102230 0x7f644019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440101870 0x7f6440196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46916/0 (socket says 172.21.15.110:46916) 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643e7fc640 1 -- 172.21.15.110:0/4246446279 learned_addr learned my addr 172.21.15.110:0/4246446279 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643effd640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6440100680 0x7f6440196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643f7fe640 1 -- 172.21.15.110:0/4246446279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6440100680 msgr2=0x7f6440196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643f7fe640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6440100680 0x7f6440196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643f7fe640 1 -- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440101870 msgr2=0x7f6440196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643f7fe640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440101870 0x7f6440196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643f7fe640 1 -- 172.21.15.110:0/4246446279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644019e530 con 0x7f6440102230 2024-04-25T00:18:57.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643e7fc640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440101870 0x7f6440196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:57.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f643f7fe640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102230 0x7f644019dec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6430009fd0 tx=0x7f643000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:57.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f6427fff640 1 -- 172.21.15.110:0/4246446279 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643000ce20 con 0x7f6440102230 2024-04-25T00:18:57.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f6427fff640 1 -- 172.21.15.110:0/4246446279 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6430005020 con 0x7f6440102230 2024-04-25T00:18:57.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644019e820 con 0x7f6440102230 2024-04-25T00:18:57.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.025+0000 7f6427fff640 1 -- 172.21.15.110:0/4246446279 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430010690 con 0x7f6440102230 2024-04-25T00:18:57.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.026+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644019edc0 con 0x7f6440102230 2024-04-25T00:18:57.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.027+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6440107b10 con 0x7f6440102230 2024-04-25T00:18:57.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.027+0000 7f6427fff640 1 -- 172.21.15.110:0/4246446279 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6430004b20 con 0x7f6440102230 2024-04-25T00:18:57.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.028+0000 7f6427fff640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64280a2c10 0x7f64280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.028+0000 7f643effd640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64280a2c10 0x7f64280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.028+0000 7f6427fff640 1 -- 172.21.15.110:0/4246446279 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f64300cf6b0 con 0x7f6440102230 2024-04-25T00:18:57.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.029+0000 7f643effd640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64280a2c10 0x7f64280a50d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6438004770 tx=0x7f6438000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:57.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.034+0000 7f6427fff640 1 -- 172.21.15.110:0/4246446279 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f643009c5e0 con 0x7f6440102230 2024-04-25T00:18:57.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.261+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6440062340 con 0x7f6440102230 2024-04-25T00:18:57.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.262+0000 7f6427fff640 1 -- 172.21.15.110:0/4246446279 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+602 (secure 0 0 0) 0x7f643009bee0 con 0x7f6440102230 2024-04-25T00:18:57.263 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2804791885 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:57.263 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1818468115 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:57.263 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3421586240 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:57.263 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3060209766 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:57.263 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/2433943234 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:57.263 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/2433943234 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:57.263 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2783100792 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:57.264 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1429818736 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:57.264 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1657096513 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:57.264 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1818468115 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:57.264 INFO:teuthology.orchestra.run.smithi110.stderr:listed 10 entries 2024-04-25T00:18:57.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64280a2c10 msgr2=0x7f64280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64280a2c10 0x7f64280a50d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6438004770 tx=0x7f6438000bc0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102230 msgr2=0x7f644019dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102230 0x7f644019dec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6430009fd0 tx=0x7f643000ef10 comp rx=0 tx=0).stop 2024-04-25T00:18:57.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 shutdown_connections 2024-04-25T00:18:57.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f64280a2c10 0x7f64280a50d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6440102230 0x7f644019dec0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6440101870 0x7f6440196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 --2- 172.21.15.110:0/4246446279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6440100680 0x7f6440196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 >> 172.21.15.110:0/4246446279 conn(0x7f64400fbbf0 msgr2=0x7f6440104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:57.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.264+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 shutdown_connections 2024-04-25T00:18:57.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.265+0000 7f64456a5640 1 -- 172.21.15.110:0/4246446279 wait complete. 2024-04-25T00:18:57.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:57 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2184516191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:57 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3856195641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:57 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:57.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:57 smithi110 ceph-mon[29187]: pgmap v858: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:18:57.275 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:57 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/197170400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.276 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T00:18:57.276 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:18:57.292 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:57 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2184516191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:57 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3856195641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:57 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:57 smithi173 ceph-mon[32402]: pgmap v858: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:18:57.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:57 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/197170400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 -- 172.21.15.110:0/2552224851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408102980 msgr2=0x7fa408104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/2552224851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408102980 0x7fa408104df0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009900 tx=0x7fa3f802a410 comp rx=0 tx=0).stop 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 -- 172.21.15.110:0/2552224851 shutdown_connections 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/2552224851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa408105330 0x7fa4081057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/2552224851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408102980 0x7fa408104df0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/2552224851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408100010 0x7fa408102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 -- 172.21.15.110:0/2552224851 >> 172.21.15.110:0/2552224851 conn(0x7fa4080fbc30 msgr2=0x7fa4080fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 -- 172.21.15.110:0/2552224851 shutdown_connections 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.438+0000 7fa40fbd5640 1 -- 172.21.15.110:0/2552224851 wait complete. 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40fbd5640 1 Processor -- start 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40fbd5640 1 -- start start 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40fbd5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408100010 0x7fa40819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40fbd5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408102980 0x7fa40819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40fbd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa408105330 0x7fa4081a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40fbd5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081138c0 con 0x7fa408100010 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40fbd5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa408113740 con 0x7fa408102980 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40fbd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa408113a40 con 0x7fa408105330 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408100010 0x7fa40819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408100010 0x7fa40819dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46922/0 (socket says 172.21.15.110:46922) 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 -- 172.21.15.110:0/3264058844 learned_addr learned my addr 172.21.15.110:0/3264058844 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d170640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408102980 0x7fa40819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40e172640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa408105330 0x7fa4081a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 -- 172.21.15.110:0/3264058844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa408105330 msgr2=0x7fa4081a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa408105330 0x7fa4081a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 -- 172.21.15.110:0/3264058844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408102980 msgr2=0x7fa40819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408102980 0x7fa40819e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 -- 172.21.15.110:0/3264058844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4081a5bf0 con 0x7fa408100010 2024-04-25T00:18:57.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.439+0000 7fa40d971640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408100010 0x7fa40819dfd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa40400d950 tx=0x7fa40400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:57.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.440+0000 7fa3feffd640 1 -- 172.21.15.110:0/3264058844 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404017070 con 0x7fa408100010 2024-04-25T00:18:57.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.440+0000 7fa3feffd640 1 -- 172.21.15.110:0/3264058844 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa404009c30 con 0x7fa408100010 2024-04-25T00:18:57.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.440+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4081a5ee0 con 0x7fa408100010 2024-04-25T00:18:57.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.440+0000 7fa3feffd640 1 -- 172.21.15.110:0/3264058844 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40400f620 con 0x7fa408100010 2024-04-25T00:18:57.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.440+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4081a6340 con 0x7fa408100010 2024-04-25T00:18:57.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.440+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4080fd720 con 0x7fa408100010 2024-04-25T00:18:57.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.442+0000 7fa3feffd640 1 -- 172.21.15.110:0/3264058844 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa4040109a0 con 0x7fa408100010 2024-04-25T00:18:57.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.442+0000 7fa3feffd640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa3f40a2b90 0x7fa3f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.442+0000 7fa3feffd640 1 -- 172.21.15.110:0/3264058844 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fa4040ceb30 con 0x7fa408100010 2024-04-25T00:18:57.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.442+0000 7fa40d170640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa3f40a2b90 0x7fa3f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.443+0000 7fa40d170640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa3f40a2b90 0x7fa3f40a5050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8006010 tx=0x7fa3f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:57.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.446+0000 7fa3feffd640 1 -- 172.21.15.110:0/3264058844 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa404009de0 con 0x7fa408100010 2024-04-25T00:18:57.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:57 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2184516191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:57 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3856195641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:57 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:57.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:57 smithi188 ceph-mon[33285]: pgmap v858: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:18:57.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:57 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/197170400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:57.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.633+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa408062370 con 0x7fa408100010 2024-04-25T00:18:57.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.634+0000 7fa3feffd640 1 -- 172.21.15.110:0/3264058844 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7fa40409b470 con 0x7fa408100010 2024-04-25T00:18:57.635 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:18:57.635 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:57.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa3f40a2b90 msgr2=0x7fa3f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa3f40a2b90 0x7fa3f40a5050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8006010 tx=0x7fa3f8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408100010 msgr2=0x7fa40819dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408100010 0x7fa40819dfd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa40400d950 tx=0x7fa40400de10 comp rx=0 tx=0).stop 2024-04-25T00:18:57.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 shutdown_connections 2024-04-25T00:18:57.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fa3f40a2b90 0x7fa3f40a5050 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa408105330 0x7fa4081a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa408102980 0x7fa40819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 --2- 172.21.15.110:0/3264058844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa408100010 0x7fa40819dfd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 >> 172.21.15.110:0/3264058844 conn(0x7fa4080fbc30 msgr2=0x7fa408100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:57.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 shutdown_connections 2024-04-25T00:18:57.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.636+0000 7fa40fbd5640 1 -- 172.21.15.110:0/3264058844 wait complete. 2024-04-25T00:18:57.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:55:22.756259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:00:01.224042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:00:02.090769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:57.648 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:18:57.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- 172.21.15.110:0/2822767994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec104c10 msgr2=0x7f71ec107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2822767994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec104c10 0x7f71ec107000 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009900 tx=0x7f71dc02a410 comp rx=0 tx=0).stop 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- 172.21.15.110:0/2822767994 shutdown_connections 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2822767994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71ec107540 0x7f71ec109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2822767994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec104c10 0x7f71ec107000 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2822767994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec1022e0 0x7f71ec1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- 172.21.15.110:0/2822767994 >> 172.21.15.110:0/2822767994 conn(0x7f71ec0fbc30 msgr2=0x7f71ec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- 172.21.15.110:0/2822767994 shutdown_connections 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- 172.21.15.110:0/2822767994 wait complete. 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 Processor -- start 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- start start 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71ec1022e0 0x7f71ec197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec104c10 0x7f71ec197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec107540 0x7f71ec19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec10d2e0 con 0x7f71ec107540 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec10d160 con 0x7f71ec104c10 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f47ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec10d460 con 0x7f71ec1022e0 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f2d8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec107540 0x7f71ec19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.749+0000 7f71f258a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71ec1022e0 0x7f71ec197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f2d8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec107540 0x7f71ec19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46930/0 (socket says 172.21.15.110:46930) 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f2d8b640 1 -- 172.21.15.110:0/2264662582 learned_addr learned my addr 172.21.15.110:0/2264662582 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f1d89640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec104c10 0x7f71ec197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f2d8b640 1 -- 172.21.15.110:0/2264662582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71ec1022e0 msgr2=0x7f71ec197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f2d8b640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71ec1022e0 0x7f71ec197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f2d8b640 1 -- 172.21.15.110:0/2264662582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec104c10 msgr2=0x7f71ec197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f2d8b640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec104c10 0x7f71ec197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f2d8b640 1 -- 172.21.15.110:0/2264662582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec19f670 con 0x7f71ec107540 2024-04-25T00:18:57.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f1d89640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec104c10 0x7f71ec197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:57.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f2d8b640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec107540 0x7f71ec19f000 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f71e800eb10 tx=0x7f71e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:57.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71e37fe640 1 -- 172.21.15.110:0/2264662582 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8004b20 con 0x7f71ec107540 2024-04-25T00:18:57.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71e37fe640 1 -- 172.21.15.110:0/2264662582 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e8004cc0 con 0x7f71ec107540 2024-04-25T00:18:57.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec19f960 con 0x7f71ec107540 2024-04-25T00:18:57.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71e37fe640 1 -- 172.21.15.110:0/2264662582 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8010690 con 0x7f71ec107540 2024-04-25T00:18:57.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.750+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec19feb0 con 0x7f71ec107540 2024-04-25T00:18:57.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.751+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec0fd720 con 0x7f71ec107540 2024-04-25T00:18:57.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.752+0000 7f71e37fe640 1 -- 172.21.15.110:0/2264662582 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f71e8005020 con 0x7f71ec107540 2024-04-25T00:18:57.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.753+0000 7f71e37fe640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f71d80a2b90 0x7f71d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:57.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.753+0000 7f71f258a640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f71d80a2b90 0x7f71d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:57.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.753+0000 7f71e37fe640 1 -- 172.21.15.110:0/2264662582 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f71e80cf780 con 0x7f71ec107540 2024-04-25T00:18:57.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.753+0000 7f71f258a640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f71d80a2b90 0x7f71d80a5050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009bd0 tx=0x7f71e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:57.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.757+0000 7f71e37fe640 1 -- 172.21.15.110:0/2264662582 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f71e809c710 con 0x7f71ec107540 2024-04-25T00:18:57.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.934+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71ec062340 con 0x7f71ec107540 2024-04-25T00:18:57.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.934+0000 7f71e37fe640 1 -- 172.21.15.110:0/2264662582 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f71e809c010 con 0x7f71ec107540 2024-04-25T00:18:57.935 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T00:18:57.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f71d80a2b90 msgr2=0x7f71d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f71d80a2b90 0x7f71d80a5050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009bd0 tx=0x7f71e4005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:57.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec107540 msgr2=0x7f71ec19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:57.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec107540 0x7f71ec19f000 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f71e800eb10 tx=0x7f71e800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 shutdown_connections 2024-04-25T00:18:57.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f71d80a2b90 0x7f71d80a5050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71ec107540 0x7f71ec19f000 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec104c10 0x7f71ec197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 --2- 172.21.15.110:0/2264662582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71ec1022e0 0x7f71ec197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:57.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.936+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 >> 172.21.15.110:0/2264662582 conn(0x7f71ec0fbc30 msgr2=0x7f71ec102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:57.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.937+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 shutdown_connections 2024-04-25T00:18:57.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:57.937+0000 7f71f47ee640 1 -- 172.21.15.110:0/2264662582 wait complete. 2024-04-25T00:18:57.948 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:18:58.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 -- 172.21.15.110:0/4016056268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 msgr2=0x7ff820108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 --2- 172.21.15.110:0/4016056268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 0x7ff820108ae0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff818009ca0 tx=0x7ff81802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 -- 172.21.15.110:0/4016056268 shutdown_connections 2024-04-25T00:18:58.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 --2- 172.21.15.110:0/4016056268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 0x7ff820108ae0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 --2- 172.21.15.110:0/4016056268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff820101890 0x7ff820101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 --2- 172.21.15.110:0/4016056268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8201006a0 0x7ff820100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 -- 172.21.15.110:0/4016056268 >> 172.21.15.110:0/4016056268 conn(0x7ff8200fbc10 msgr2=0x7ff8200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 -- 172.21.15.110:0/4016056268 shutdown_connections 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.049+0000 7ff828438640 1 -- 172.21.15.110:0/4016056268 wait complete. 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff828438640 1 Processor -- start 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff828438640 1 -- start start 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff828438640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8201006a0 0x7ff82006a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff828438640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff820101890 0x7ff82019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff828438640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 0x7ff82019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff828438640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010c1b0 con 0x7ff820102250 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff828438640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010c030 con 0x7ff8201006a0 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff828438640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010c330 con 0x7ff820101890 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 0x7ff82019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 0x7ff82019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46940/0 (socket says 172.21.15.110:46940) 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 -- 172.21.15.110:0/527932132 learned_addr learned my addr 172.21.15.110:0/527932132 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8261d4640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8201006a0 0x7ff82006a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8259d3640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff820101890 0x7ff82019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 -- 172.21.15.110:0/527932132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff820101890 msgr2=0x7ff82019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff820101890 0x7ff82019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 -- 172.21.15.110:0/527932132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8201006a0 msgr2=0x7ff82006a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8201006a0 0x7ff82006a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 -- 172.21.15.110:0/527932132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82019f600 con 0x7ff820102250 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8261d4640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8201006a0 0x7ff82006a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8269d5640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 0x7ff82019ecf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff818031970 tx=0x7ff818003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.050+0000 7ff8177fe640 1 -- 172.21.15.110:0/527932132 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818004040 con 0x7ff820102250 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.051+0000 7ff8177fe640 1 -- 172.21.15.110:0/527932132 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff818004690 con 0x7ff820102250 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.051+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82019f830 con 0x7ff820102250 2024-04-25T00:18:58.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.051+0000 7ff8177fe640 1 -- 172.21.15.110:0/527932132 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818030c60 con 0x7ff820102250 2024-04-25T00:18:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.051+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82019fd80 con 0x7ff820102250 2024-04-25T00:18:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.051+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff820107a90 con 0x7ff820102250 2024-04-25T00:18:58.055 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.052+0000 7ff8177fe640 1 -- 172.21.15.110:0/527932132 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff8180041e0 con 0x7ff820102250 2024-04-25T00:18:58.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.053+0000 7ff8177fe640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff80c0a2b90 0x7ff80c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.053+0000 7ff8261d4640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff80c0a2b90 0x7ff80c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.054+0000 7ff8177fe640 1 -- 172.21.15.110:0/527932132 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7ff8180f0670 con 0x7ff820102250 2024-04-25T00:18:58.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.054+0000 7ff8261d4640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff80c0a2b90 0x7ff80c0a5050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff8201016f0 tx=0x7ff81c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:58.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.057+0000 7ff8177fe640 1 -- 172.21.15.110:0/527932132 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8180bd5a0 con 0x7ff820102250 2024-04-25T00:18:58.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.230+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff82002d300 con 0x7ff820102250 2024-04-25T00:18:58.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.230+0000 7ff8177fe640 1 -- 172.21.15.110:0/527932132 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7ff8180bcea0 con 0x7ff820102250 2024-04-25T00:18:58.321 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T00:18:58.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff80c0a2b90 msgr2=0x7ff80c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff80c0a2b90 0x7ff80c0a5050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff8201016f0 tx=0x7ff81c00b390 comp rx=0 tx=0).stop 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 msgr2=0x7ff82019ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 0x7ff82019ecf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff818031970 tx=0x7ff818003ba0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 shutdown_connections 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff80c0a2b90 0x7ff80c0a5050 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff820102250 0x7ff82019ecf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff820101890 0x7ff82019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 --2- 172.21.15.110:0/527932132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8201006a0 0x7ff82006a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.232+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 >> 172.21.15.110:0/527932132 conn(0x7ff8200fbc10 msgr2=0x7ff820104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.233+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 shutdown_connections 2024-04-25T00:18:58.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.233+0000 7ff828438640 1 -- 172.21.15.110:0/527932132 wait complete. 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4246446279' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3264058844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:58.324 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:58 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2264662582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:58.325 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:18:58.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4246446279' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3264058844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:58.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:58 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2264662582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:58.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.425+0000 7f515c73b640 1 -- 172.21.15.110:0/98687275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154102990 msgr2=0x7f5154102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.425+0000 7f515c73b640 1 --2- 172.21.15.110:0/98687275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154102990 0x7f5154102e10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5144009900 tx=0x7f514402a410 comp rx=0 tx=0).stop 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 -- 172.21.15.110:0/98687275 shutdown_connections 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 --2- 172.21.15.110:0/98687275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5154103350 0x7f5154109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 --2- 172.21.15.110:0/98687275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154102990 0x7f5154102e10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 --2- 172.21.15.110:0/98687275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51541017a0 0x7f5154101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 -- 172.21.15.110:0/98687275 >> 172.21.15.110:0/98687275 conn(0x7f51540fcd10 msgr2=0x7f51540ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 -- 172.21.15.110:0/98687275 shutdown_connections 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 -- 172.21.15.110:0/98687275 wait complete. 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 Processor -- start 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 -- start start 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51541017a0 0x7f5154197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5154102990 0x7f5154197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154103350 0x7f515419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d2b0 con 0x7f5154103350 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d130 con 0x7f51541017a0 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515c73b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d430 con 0x7f5154102990 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515acd8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154103350 0x7f515419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515acd8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154103350 0x7f515419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46960/0 (socket says 172.21.15.110:46960) 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.426+0000 7f515acd8640 1 -- 172.21.15.110:0/4173657986 learned_addr learned my addr 172.21.15.110:0/4173657986 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515a4d7640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51541017a0 0x7f5154197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515acd8640 1 -- 172.21.15.110:0/4173657986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5154102990 msgr2=0x7f5154197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f5159cd6640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5154102990 0x7f5154197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515acd8640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5154102990 0x7f5154197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515acd8640 1 -- 172.21.15.110:0/4173657986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51541017a0 msgr2=0x7f5154197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515acd8640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51541017a0 0x7f5154197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515acd8640 1 -- 172.21.15.110:0/4173657986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515419f570 con 0x7f5154103350 2024-04-25T00:18:58.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515acd8640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154103350 0x7f515419ef00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f514c00eaa0 tx=0x7f514c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:58.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f514b7fe640 1 -- 172.21.15.110:0/4173657986 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c00cdb0 con 0x7f5154103350 2024-04-25T00:18:58.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515419f860 con 0x7f5154103350 2024-04-25T00:18:58.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f514b7fe640 1 -- 172.21.15.110:0/4173657986 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f514c005020 con 0x7f5154103350 2024-04-25T00:18:58.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f514b7fe640 1 -- 172.21.15.110:0/4173657986 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c0116d0 con 0x7f5154103350 2024-04-25T00:18:58.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.427+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515419fd60 con 0x7f5154103350 2024-04-25T00:18:58.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.428+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5154108b40 con 0x7f5154103350 2024-04-25T00:18:58.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.429+0000 7f514b7fe640 1 -- 172.21.15.110:0/4173657986 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f514c0051d0 con 0x7f5154103350 2024-04-25T00:18:58.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.429+0000 7f514b7fe640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f51400a2b90 0x7f51400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.430+0000 7f515a4d7640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f51400a2b90 0x7f51400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.430+0000 7f514b7fe640 1 -- 172.21.15.110:0/4173657986 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f514c0cf2d0 con 0x7f5154103350 2024-04-25T00:18:58.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.430+0000 7f515a4d7640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f51400a2b90 0x7f51400a5050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51500058c0 tx=0x7f5150005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:58.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.434+0000 7f514b7fe640 1 -- 172.21.15.110:0/4173657986 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f514c016030 con 0x7f5154103350 2024-04-25T00:18:58.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4246446279' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:18:58.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:58.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-04-25T00:18:58.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T00:18:58.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:58.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:58.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' 2024-04-25T00:18:58.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3264058844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:58.519 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:58 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2264662582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:58.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.613+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51540623e0 con 0x7f5154103350 2024-04-25T00:18:58.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.613+0000 7f514b7fe640 1 -- 172.21.15.110:0/4173657986 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f514c09c260 con 0x7f5154103350 2024-04-25T00:18:58.614 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 64 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f51400a2b90 msgr2=0x7f51400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f51400a2b90 0x7f51400a5050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51500058c0 tx=0x7f5150005850 comp rx=0 tx=0).stop 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154103350 msgr2=0x7f515419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154103350 0x7f515419ef00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f514c00eaa0 tx=0x7f514c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 shutdown_connections 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f51400a2b90 0x7f51400a5050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5154103350 0x7f515419ef00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5154102990 0x7f5154197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 --2- 172.21.15.110:0/4173657986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51541017a0 0x7f5154197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 >> 172.21.15.110:0/4173657986 conn(0x7f51540fcd10 msgr2=0x7f5154105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:58.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 shutdown_connections 2024-04-25T00:18:58.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.615+0000 7f515c73b640 1 -- 172.21.15.110:0/4173657986 wait complete. 2024-04-25T00:18:58.627 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:18:58.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 -- 172.21.15.110:0/1414378276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c104b30 msgr2=0x7f471c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 --2- 172.21.15.110:0/1414378276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c104b30 0x7f471c106f20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f470c009900 tx=0x7f470c02a410 comp rx=0 tx=0).stop 2024-04-25T00:18:58.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 -- 172.21.15.110:0/1414378276 shutdown_connections 2024-04-25T00:18:58.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 --2- 172.21.15.110:0/1414378276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f471c107460 0x7f471c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 --2- 172.21.15.110:0/1414378276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c104b30 0x7f471c106f20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 --2- 172.21.15.110:0/1414378276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f471c102200 0x7f471c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 -- 172.21.15.110:0/1414378276 >> 172.21.15.110:0/1414378276 conn(0x7f471c0fbc30 msgr2=0x7f471c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 -- 172.21.15.110:0/1414378276 shutdown_connections 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.725+0000 7f47242df640 1 -- 172.21.15.110:0/1414378276 wait complete. 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 Processor -- start 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 -- start start 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c102200 0x7f471c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f471c104b30 0x7f471c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f471c107460 0x7f471c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f471c10d2b0 con 0x7f471c102200 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f471c10d130 con 0x7f471c104b30 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f471c10d430 con 0x7f471c107460 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c102200 0x7f471c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c102200 0x7f471c197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46976/0 (socket says 172.21.15.110:46976) 2024-04-25T00:18:58.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 -- 172.21.15.110:0/3598795330 learned_addr learned my addr 172.21.15.110:0/3598795330 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472187a640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f471c104b30 0x7f471c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472287c640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f471c107460 0x7f471c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 -- 172.21.15.110:0/3598795330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f471c107460 msgr2=0x7f471c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f471c107460 0x7f471c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 -- 172.21.15.110:0/3598795330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f471c104b30 msgr2=0x7f471c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f471c104b30 0x7f471c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 -- 172.21.15.110:0/3598795330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c19f640 con 0x7f471c102200 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472287c640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f471c107460 0x7f471c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f472207b640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c102200 0x7f471c197a20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f471800d950 tx=0x7f471800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47137fe640 1 -- 172.21.15.110:0/3598795330 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718017070 con 0x7f471c102200 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47137fe640 1 -- 172.21.15.110:0/3598795330 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4718009c30 con 0x7f471c102200 2024-04-25T00:18:58.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471c19f930 con 0x7f471c102200 2024-04-25T00:18:58.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.726+0000 7f47137fe640 1 -- 172.21.15.110:0/3598795330 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471800f620 con 0x7f471c102200 2024-04-25T00:18:58.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.727+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471c19fe50 con 0x7f471c102200 2024-04-25T00:18:58.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.728+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471c0fd720 con 0x7f471c102200 2024-04-25T00:18:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.729+0000 7f47137fe640 1 -- 172.21.15.110:0/3598795330 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f47180109a0 con 0x7f471c102200 2024-04-25T00:18:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.729+0000 7f47137fe640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f47080a2b90 0x7f47080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.729+0000 7f472187a640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f47080a2b90 0x7f47080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.730+0000 7f47137fe640 1 -- 172.21.15.110:0/3598795330 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f47180cec60 con 0x7f471c102200 2024-04-25T00:18:58.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.730+0000 7f472187a640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f47080a2b90 0x7f47080a5050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f470c004770 tx=0x7f470c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:58.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.734+0000 7f47137fe640 1 -- 172.21.15.110:0/3598795330 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4718009de0 con 0x7f471c102200 2024-04-25T00:18:58.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.923+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f471c02d170 con 0x7f471c102200 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.923+0000 7f47137fe640 1 -- 172.21.15.110:0/3598795330 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+602 (secure 0 0 0) 0x7f471809bbf0 con 0x7f471c102200 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2804791885 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1818468115 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3421586240 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3060209766 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/2433943234 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/2433943234 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2783100792 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:58.924 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1429818736 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:58.925 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1657096513 2024-04-25T23:52:27.865188+0000 2024-04-25T00:18:58.925 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1818468115 2024-04-25T23:52:54.426106+0000 2024-04-25T00:18:58.925 INFO:teuthology.orchestra.run.smithi110.stderr:listed 10 entries 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f47080a2b90 msgr2=0x7f47080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f47080a2b90 0x7f47080a5050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f470c004770 tx=0x7f470c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c102200 msgr2=0x7f471c197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c102200 0x7f471c197a20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f471800d950 tx=0x7f471800de10 comp rx=0 tx=0).stop 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 shutdown_connections 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f47080a2b90 0x7f47080a5050 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f471c107460 0x7f471c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f471c104b30 0x7f471c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 --2- 172.21.15.110:0/3598795330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f471c102200 0x7f471c197a20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 >> 172.21.15.110:0/3598795330 conn(0x7f471c0fbc30 msgr2=0x7f471c102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:58.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 shutdown_connections 2024-04-25T00:18:58.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:18:58.925+0000 7f47242df640 1 -- 172.21.15.110:0/3598795330 wait complete. 2024-04-25T00:18:58.937 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi110.front.sepia.ceph.com... 2024-04-25T00:18:58.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:58.937 DEBUG:teuthology.orchestra.run.smithi110:> sudo fusermount -u /tmp/cephfs 2024-04-25T00:18:58.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi110.stderr:ceph-fuse[80833]: fuse finished with error 0 and tester_r 0 2024-04-25T00:18:59.070 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:18:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:59 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/527932132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:59 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4173657986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:59 smithi188 ceph-mon[33285]: pgmap v859: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-04-25T00:18:59.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:18:59 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3598795330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:18:59.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:59 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/527932132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:59.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:59 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4173657986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:59.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:59 smithi110 ceph-mon[29187]: pgmap v859: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-04-25T00:18:59.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:18:59 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3598795330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:18:59.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:59 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/527932132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:59 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4173657986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:59 smithi173 ceph-mon[32402]: pgmap v859: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-04-25T00:18:59.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:18:59 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3598795330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:19:01.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:00 smithi188 ceph-mon[33285]: pgmap v860: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:19:01.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:00 smithi110 ceph-mon[29187]: pgmap v860: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:19:01.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:00 smithi173 ceph-mon[32402]: pgmap v860: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T00:19:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:02 smithi188 ceph-mon[33285]: pgmap v861: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-04-25T00:19:03.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:02 smithi110 ceph-mon[29187]: pgmap v861: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-04-25T00:19:03.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:02 smithi173 ceph-mon[32402]: pgmap v861: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-04-25T00:19:05.172 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T00:19:05.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:05.172 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-04-25T00:19:05.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:05.215 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-04-25T00:19:05.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:04 smithi188 ceph-mon[33285]: pgmap v862: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 6 op/s 2024-04-25T00:19:05.280 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-04-25T00:19:05.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:05.280 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-04-25T00:19:05.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:04 smithi110 ceph-mon[29187]: pgmap v862: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 6 op/s 2024-04-25T00:19:05.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:05.314 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-04-25T00:19:05.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:04 smithi173 ceph-mon[32402]: pgmap v862: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 6 op/s 2024-04-25T00:19:05.413 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T00:19:05.424 INFO:tasks.cephadm:Teardown begin 2024-04-25T00:19:05.424 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:19:05.424 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:05.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 -- 172.21.15.110:0/1761011137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc100680 msgr2=0x7f20cc100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:05.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 --2- 172.21.15.110:0/1761011137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc100680 0x7f20cc100aa0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009900 tx=0x7f20bc02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:05.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 -- 172.21.15.110:0/1761011137 shutdown_connections 2024-04-25T00:19:05.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 --2- 172.21.15.110:0/1761011137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20cc102230 0x7f20cc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 --2- 172.21.15.110:0/1761011137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20cc101870 0x7f20cc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 --2- 172.21.15.110:0/1761011137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc100680 0x7f20cc100aa0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 -- 172.21.15.110:0/1761011137 >> 172.21.15.110:0/1761011137 conn(0x7f20cc0fbbf0 msgr2=0x7f20cc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 -- 172.21.15.110:0/1761011137 shutdown_connections 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 -- 172.21.15.110:0/1761011137 wait complete. 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 Processor -- start 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.527+0000 7f20d2812640 1 -- start start 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d2812640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20cc100680 0x7f20cc069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d2812640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20cc101870 0x7f20cc19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d2812640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc102230 0x7f20cc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d2812640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc10c180 con 0x7f20cc102230 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d2812640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc10c000 con 0x7f20cc101870 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d2812640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc10c300 con 0x7f20cc100680 2024-04-25T00:19:05.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc102230 0x7f20cc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc102230 0x7f20cc19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47000/0 (socket says 172.21.15.110:47000) 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 -- 172.21.15.110:0/633745701 learned_addr learned my addr 172.21.15.110:0/633745701 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20cbfff640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20cc100680 0x7f20cc069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20cb7fe640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20cc101870 0x7f20cc19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 -- 172.21.15.110:0/633745701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20cc100680 msgr2=0x7f20cc069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20cc100680 0x7f20cc069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 -- 172.21.15.110:0/633745701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20cc101870 msgr2=0x7f20cc19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20cc101870 0x7f20cc19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 -- 172.21.15.110:0/633745701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc19f6b0 con 0x7f20cc102230 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20cb7fe640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20cc101870 0x7f20cc19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d0daf640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc102230 0x7f20cc19eda0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f20c400cad0 tx=0x7f20c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:05.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20c97fa640 1 -- 172.21.15.110:0/633745701 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400ae20 con 0x7f20cc102230 2024-04-25T00:19:05.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20c97fa640 1 -- 172.21.15.110:0/633745701 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20c4005020 con 0x7f20cc102230 2024-04-25T00:19:05.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc19f940 con 0x7f20cc102230 2024-04-25T00:19:05.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20c97fa640 1 -- 172.21.15.110:0/633745701 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400f6b0 con 0x7f20cc102230 2024-04-25T00:19:05.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.528+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc19fe40 con 0x7f20cc102230 2024-04-25T00:19:05.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.529+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20cc107ac0 con 0x7f20cc102230 2024-04-25T00:19:05.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.531+0000 7f20c97fa640 1 -- 172.21.15.110:0/633745701 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f20c40051d0 con 0x7f20cc102230 2024-04-25T00:19:05.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.531+0000 7f20c97fa640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f20b40a2b90 0x7f20b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:05.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.531+0000 7f20cbfff640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f20b40a2b90 0x7f20b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:05.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.531+0000 7f20c97fa640 1 -- 172.21.15.110:0/633745701 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f20c40cf4b0 con 0x7f20cc102230 2024-04-25T00:19:05.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.532+0000 7f20cbfff640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f20b40a2b90 0x7f20b40a5050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20bc000c00 tx=0x7f20bc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:05.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.535+0000 7f20c97fa640 1 -- 172.21.15.110:0/633745701 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f20c4014030 con 0x7f20cc102230 2024-04-25T00:19:05.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.747+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20cc02d170 con 0x7f20cc102230 2024-04-25T00:19:05.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.748+0000 7f20c97fa640 1 -- 172.21.15.110:0/633745701 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11884 (secure 0 0 0) 0x7f20c409c440 con 0x7f20cc102230 2024-04-25T00:19:05.748 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:19:05.748 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:00:02.856564+0000","modified":"2024-04-25T00:07:34.060466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24476,24512,24457]},"id":1}]} 2024-04-25T00:19:05.749 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 32 2024-04-25T00:19:05.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f20b40a2b90 msgr2=0x7f20b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:05.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f20b40a2b90 0x7f20b40a5050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20bc000c00 tx=0x7f20bc0314a0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc102230 msgr2=0x7f20cc19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc102230 0x7f20cc19eda0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f20c400cad0 tx=0x7f20c400cf90 comp rx=0 tx=0).stop 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 shutdown_connections 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f20b40a2b90 0x7f20b40a5050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20cc102230 0x7f20cc19eda0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20cc101870 0x7f20cc19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 --2- 172.21.15.110:0/633745701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20cc100680 0x7f20cc069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 >> 172.21.15.110:0/633745701 conn(0x7f20cc0fbbf0 msgr2=0x7f20cc104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 shutdown_connections 2024-04-25T00:19:05.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.750+0000 7f20d2812640 1 -- 172.21.15.110:0/633745701 wait complete. 2024-04-25T00:19:05.762 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 -- 172.21.15.110:0/1152399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468102280 msgr2=0x7fc468104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 --2- 172.21.15.110:0/1152399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468102280 0x7fc468104670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc464009900 tx=0x7fc46402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 -- 172.21.15.110:0/1152399 shutdown_connections 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 --2- 172.21.15.110:0/1152399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4681074e0 0x7fc4681098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 --2- 172.21.15.110:0/1152399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc468104bb0 0x7fc468106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 --2- 172.21.15.110:0/1152399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468102280 0x7fc468104670 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 -- 172.21.15.110:0/1152399 >> 172.21.15.110:0/1152399 conn(0x7fc4680fbc30 msgr2=0x7fc4680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 -- 172.21.15.110:0/1152399 shutdown_connections 2024-04-25T00:19:05.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.862+0000 7fc46fec3640 1 -- 172.21.15.110:0/1152399 wait complete. 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46fec3640 1 Processor -- start 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46fec3640 1 -- start start 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46fec3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc468102280 0x7fc468197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46fec3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468104bb0 0x7fc468197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46fec3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4681074e0 0x7fc46819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46fec3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810d2f0 con 0x7fc468104bb0 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46fec3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810d170 con 0x7fc4681074e0 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46fec3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810d470 con 0x7fc468102280 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46d45e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468104bb0 0x7fc468197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46d45e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468104bb0 0x7fc468197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47024/0 (socket says 172.21.15.110:47024) 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46d45e640 1 -- 172.21.15.110:0/4283849877 learned_addr learned my addr 172.21.15.110:0/4283849877 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:19:05.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46dc5f640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc468102280 0x7fc468197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46e460640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4681074e0 0x7fc46819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46d45e640 1 -- 172.21.15.110:0/4283849877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc468102280 msgr2=0x7fc468197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46d45e640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc468102280 0x7fc468197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46d45e640 1 -- 172.21.15.110:0/4283849877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4681074e0 msgr2=0x7fc46819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46d45e640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4681074e0 0x7fc46819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46d45e640 1 -- 172.21.15.110:0/4283849877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46819f680 con 0x7fc468104bb0 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.863+0000 7fc46e460640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4681074e0 0x7fc46819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.864+0000 7fc46d45e640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468104bb0 0x7fc468197fa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d740 tx=0x7fc45800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.864+0000 7fc45effd640 1 -- 172.21.15.110:0/4283849877 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458017070 con 0x7fc468104bb0 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.864+0000 7fc45effd640 1 -- 172.21.15.110:0/4283849877 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc45800de50 con 0x7fc468104bb0 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.864+0000 7fc45effd640 1 -- 172.21.15.110:0/4283849877 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458004d60 con 0x7fc468104bb0 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.864+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46819f970 con 0x7fc468104bb0 2024-04-25T00:19:05.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.864+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46819fe70 con 0x7fc468104bb0 2024-04-25T00:19:05.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.865+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4680fd720 con 0x7fc468104bb0 2024-04-25T00:19:05.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.866+0000 7fc45effd640 1 -- 172.21.15.110:0/4283849877 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc458004f00 con 0x7fc468104bb0 2024-04-25T00:19:05.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.866+0000 7fc45effd640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4540a2b90 0x7fc4540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:05.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.867+0000 7fc46dc5f640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4540a2b90 0x7fc4540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:05.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.867+0000 7fc45effd640 1 -- 172.21.15.110:0/4283849877 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fc4580ceda0 con 0x7fc468104bb0 2024-04-25T00:19:05.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.867+0000 7fc46dc5f640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4540a2b90 0x7fc4540a5050 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc464006010 tx=0x7fc464000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:05.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:05.870+0000 7fc45effd640 1 -- 172.21.15.110:0/4283849877 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc45809bd30 con 0x7fc468104bb0 2024-04-25T00:19:06.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:05 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/633745701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:06.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.090+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc46802d170 con 0x7fc468104bb0 2024-04-25T00:19:06.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.091+0000 7fc45effd640 1 -- 172.21.15.110:0/4283849877 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11884 (secure 0 0 0) 0x7fc45809b630 con 0x7fc468104bb0 2024-04-25T00:19:06.091 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:19:06.092 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1261890335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1261890335},{"type":"v1","addr":"172.21.15.110:6835","nonce":1261890335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/1723817194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":1723817194},{"type":"v1","addr":"172.21.15.110:6837","nonce":1723817194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/717154296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":717154296},{"type":"v1","addr":"172.21.15.110:6839","nonce":717154296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6841/2788119142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6840","nonce":2788119142},{"type":"v1","addr":"172.21.15.110:6841","nonce":2788119142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2454659277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2454659277},{"type":"v1","addr":"172.21.15.188:6833","nonce":2454659277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/373959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":373959969},{"type":"v1","addr":"172.21.15.173:6833","nonce":373959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:00:02.856564+0000","modified":"2024-04-25T00:07:34.060466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24476,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/946560580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":946560580},{"type":"v1","addr":"172.21.15.188:6835","nonce":946560580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/1627462282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1627462282},{"type":"v1","addr":"172.21.15.173:6835","nonce":1627462282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/1170231083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1170231083},{"type":"v1","addr":"172.21.15.188:6837","nonce":1170231083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6839/361433277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":361433277},{"type":"v1","addr":"172.21.15.188:6839","nonce":361433277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6837/3367670345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3367670345},{"type":"v1","addr":"172.21.15.173:6837","nonce":3367670345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2519479764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2519479764},{"type":"v1","addr":"172.21.15.173:6839","nonce":2519479764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24476,24512,24457]},"id":1}]} 2024-04-25T00:19:06.092 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 32 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4540a2b90 msgr2=0x7fc4540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4540a2b90 0x7fc4540a5050 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc464006010 tx=0x7fc464000bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468104bb0 msgr2=0x7fc468197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468104bb0 0x7fc468197fa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d740 tx=0x7fc45800dc00 comp rx=0 tx=0).stop 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 shutdown_connections 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fc4540a2b90 0x7fc4540a5050 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4681074e0 0x7fc46819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc468104bb0 0x7fc468197fa0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 --2- 172.21.15.110:0/4283849877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc468102280 0x7fc468197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 >> 172.21.15.110:0/4283849877 conn(0x7fc4680fbc30 msgr2=0x7fc468102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 shutdown_connections 2024-04-25T00:19:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.093+0000 7fc46fec3640 1 -- 172.21.15.110:0/4283849877 wait complete. 2024-04-25T00:19:06.105 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:06.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 -- 172.21.15.110:0/3485557999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 msgr2=0x7f5fec101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/3485557999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 0x7f5fec101d10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009900 tx=0x7f5fd802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 -- 172.21.15.110:0/3485557999 shutdown_connections 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/3485557999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fec102250 0x7f5fec108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/3485557999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 0x7f5fec101d10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/3485557999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fec1006a0 0x7f5fec100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 -- 172.21.15.110:0/3485557999 >> 172.21.15.110:0/3485557999 conn(0x7f5fec0fbc30 msgr2=0x7f5fec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 -- 172.21.15.110:0/3485557999 shutdown_connections 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.201+0000 7f5ff2b63640 1 -- 172.21.15.110:0/3485557999 wait complete. 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff2b63640 1 Processor -- start 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff2b63640 1 -- start start 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff2b63640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fec1006a0 0x7f5fec1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff2b63640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 0x7f5fec197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff2b63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fec102250 0x7f5fec19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff2b63640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec10d2f0 con 0x7f5fec101890 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff2b63640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec10d170 con 0x7f5fec1006a0 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff2b63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec10d470 con 0x7f5fec102250 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5febfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 0x7f5fec197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5febfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 0x7f5fec197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47044/0 (socket says 172.21.15.110:47044) 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5febfff640 1 -- 172.21.15.110:0/1234015088 learned_addr learned my addr 172.21.15.110:0/1234015088 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff08ff640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fec1006a0 0x7f5fec1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5ff1100640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fec102250 0x7f5fec19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5febfff640 1 -- 172.21.15.110:0/1234015088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fec102250 msgr2=0x7f5fec19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5febfff640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fec102250 0x7f5fec19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5febfff640 1 -- 172.21.15.110:0/1234015088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fec1006a0 msgr2=0x7f5fec1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5febfff640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fec1006a0 0x7f5fec1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.202+0000 7f5febfff640 1 -- 172.21.15.110:0/1234015088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec19f5f0 con 0x7f5fec101890 2024-04-25T00:19:06.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.203+0000 7f5febfff640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 0x7f5fec197f10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005940 tx=0x7f5fd8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:06.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.203+0000 7f5fe9ffb640 1 -- 172.21.15.110:0/1234015088 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd803f030 con 0x7f5fec101890 2024-04-25T00:19:06.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.203+0000 7f5fe9ffb640 1 -- 172.21.15.110:0/1234015088 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fd8003cb0 con 0x7f5fec101890 2024-04-25T00:19:06.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.203+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec19f880 con 0x7f5fec101890 2024-04-25T00:19:06.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.203+0000 7f5fe9ffb640 1 -- 172.21.15.110:0/1234015088 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8035070 con 0x7f5fec101890 2024-04-25T00:19:06.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.203+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec19fdc0 con 0x7f5fec101890 2024-04-25T00:19:06.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.204+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fec107a60 con 0x7f5fec101890 2024-04-25T00:19:06.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.205+0000 7f5fe9ffb640 1 -- 172.21.15.110:0/1234015088 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5fd8042030 con 0x7f5fec101890 2024-04-25T00:19:06.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.205+0000 7f5fe9ffb640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fdc0a2c10 0x7f5fdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.206+0000 7f5fe9ffb640 1 -- 172.21.15.110:0/1234015088 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7f5fd80efcb0 con 0x7f5fec101890 2024-04-25T00:19:06.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.206+0000 7f5ff08ff640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fdc0a2c10 0x7f5fdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.206+0000 7f5ff08ff640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fdc0a2c10 0x7f5fdc0a50d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5fec1016f0 tx=0x7f5fe000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:06.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.209+0000 7f5fe9ffb640 1 -- 172.21.15.110:0/1234015088 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5fd80bcc40 con 0x7f5fec101890 2024-04-25T00:19:06.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:05 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/633745701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:06.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:05 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/633745701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:06.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.386+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fec02d330 con 0x7f5fec101890 2024-04-25T00:19:06.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.387+0000 7f5fe9ffb640 1 -- 172.21.15.110:0/1234015088 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22653 (secure 0 0 0) 0x7f5fd80bc540 con 0x7f5fec101890 2024-04-25T00:19:06.388 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T00:19:06.388 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":87,"fsid":"787fa75a-0295-11ef-bc93-c7b262605968","created":"2024-04-24T23:51:53.430143+0000","modified":"2024-04-25T00:00:55.648110+0000","last_up_change":"2024-04-24T23:59:12.651900+0000","last_in_change":"2024-04-24T23:58:52.770619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:55:22.756259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:00:01.224042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:00:02.090769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e6eba556-dc44-4043-a0d7-c9d6dc839473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6803","nonce":1941809226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6805","nonce":1941809226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6809","nonce":1941809226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1941809226},{"type":"v1","addr":"172.21.15.110:6807","nonce":1941809226}]},"public_addr":"172.21.15.110:6803/1941809226","cluster_addr":"172.21.15.110:6805/1941809226","heartbeat_back_addr":"172.21.15.110:6809/1941809226","heartbeat_front_addr":"172.21.15.110:6807/1941809226","state":["exists","up"]},{"osd":1,"uuid":"fbfc4554-e9a2-4cc3-a392-cfe42b675b63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6801","nonce":1633800415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6803","nonce":1633800415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6807","nonce":1633800415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1633800415},{"type":"v1","addr":"172.21.15.173:6805","nonce":1633800415}]},"public_addr":"172.21.15.173:6801/1633800415","cluster_addr":"172.21.15.173:6803/1633800415","heartbeat_back_addr":"172.21.15.173:6807/1633800415","heartbeat_front_addr":"172.21.15.173:6805/1633800415","state":["exists","up"]},{"osd":2,"uuid":"3c3e4437-7088-4060-a771-70bebf386247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6801","nonce":2140926702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6803","nonce":2140926702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6807","nonce":2140926702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2140926702},{"type":"v1","addr":"172.21.15.188:6805","nonce":2140926702}]},"public_addr":"172.21.15.188:6801/2140926702","cluster_addr":"172.21.15.188:6803/2140926702","heartbeat_back_addr":"172.21.15.188:6807/2140926702","heartbeat_front_addr":"172.21.15.188:6805/2140926702","state":["exists","up"]},{"osd":3,"uuid":"307060eb-b8bb-4b80-b43e-cc22c933400f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6811","nonce":2506639922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6813","nonce":2506639922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6817","nonce":2506639922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":2506639922},{"type":"v1","addr":"172.21.15.110:6815","nonce":2506639922}]},"public_addr":"172.21.15.110:6811/2506639922","cluster_addr":"172.21.15.110:6813/2506639922","heartbeat_back_addr":"172.21.15.110:6817/2506639922","heartbeat_front_addr":"172.21.15.110:6815/2506639922","state":["exists","up"]},{"osd":4,"uuid":"12c128f9-0d7a-44ff-96c1-b2e8417851d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6809","nonce":3034286284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6811","nonce":3034286284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6815","nonce":3034286284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3034286284},{"type":"v1","addr":"172.21.15.173:6813","nonce":3034286284}]},"public_addr":"172.21.15.173:6809/3034286284","cluster_addr":"172.21.15.173:6811/3034286284","heartbeat_back_addr":"172.21.15.173:6815/3034286284","heartbeat_front_addr":"172.21.15.173:6813/3034286284","state":["exists","up"]},{"osd":5,"uuid":"6b479fe2-2120-4b2c-a0d6-986d5d7582d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6809","nonce":216258077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6811","nonce":216258077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6815","nonce":216258077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":216258077},{"type":"v1","addr":"172.21.15.188:6813","nonce":216258077}]},"public_addr":"172.21.15.188:6809/216258077","cluster_addr":"172.21.15.188:6811/216258077","heartbeat_back_addr":"172.21.15.188:6815/216258077","heartbeat_front_addr":"172.21.15.188:6813/216258077","state":["exists","up"]},{"osd":6,"uuid":"9bf7fd3c-3d98-411e-b50b-bbe5ea52f664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6819","nonce":3643107507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6821","nonce":3643107507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6825","nonce":3643107507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3643107507},{"type":"v1","addr":"172.21.15.110:6823","nonce":3643107507}]},"public_addr":"172.21.15.110:6819/3643107507","cluster_addr":"172.21.15.110:6821/3643107507","heartbeat_back_addr":"172.21.15.110:6825/3643107507","heartbeat_front_addr":"172.21.15.110:6823/3643107507","state":["exists","up"]},{"osd":7,"uuid":"19d1620c-d00e-4c4a-b0a3-acf84b3021ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6817","nonce":3085868439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6819","nonce":3085868439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6823","nonce":3085868439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3085868439},{"type":"v1","addr":"172.21.15.173:6821","nonce":3085868439}]},"public_addr":"172.21.15.173:6817/3085868439","cluster_addr":"172.21.15.173:6819/3085868439","heartbeat_back_addr":"172.21.15.173:6823/3085868439","heartbeat_front_addr":"172.21.15.173:6821/3085868439","state":["exists","up"]},{"osd":8,"uuid":"b06d6f8f-9a4c-4113-91a1-0e497dc0aaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6817","nonce":879002066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6819","nonce":879002066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6823","nonce":879002066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":879002066},{"type":"v1","addr":"172.21.15.188:6821","nonce":879002066}]},"public_addr":"172.21.15.188:6817/879002066","cluster_addr":"172.21.15.188:6819/879002066","heartbeat_back_addr":"172.21.15.188:6823/879002066","heartbeat_front_addr":"172.21.15.188:6821/879002066","state":["exists","up"]},{"osd":9,"uuid":"1d5cf0eb-8bf8-488a-8183-f8c9961006f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6827","nonce":453265858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6829","nonce":453265858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6833","nonce":453265858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":453265858},{"type":"v1","addr":"172.21.15.110:6831","nonce":453265858}]},"public_addr":"172.21.15.110:6827/453265858","cluster_addr":"172.21.15.110:6829/453265858","heartbeat_back_addr":"172.21.15.110:6833/453265858","heartbeat_front_addr":"172.21.15.110:6831/453265858","state":["exists","up"]},{"osd":10,"uuid":"6319a25f-1d2b-404a-8dd4-ff9fa9dc2a51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6825","nonce":3803887034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6827","nonce":3803887034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6831","nonce":3803887034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":3803887034},{"type":"v1","addr":"172.21.15.173:6829","nonce":3803887034}]},"public_addr":"172.21.15.173:6825/3803887034","cluster_addr":"172.21.15.173:6827/3803887034","heartbeat_back_addr":"172.21.15.173:6831/3803887034","heartbeat_front_addr":"172.21.15.173:6829/3803887034","state":["exists","up"]},{"osd":11,"uuid":"1f2ae40f-4939-417d-b66f-990914900d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6825","nonce":785596824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6827","nonce":785596824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6831","nonce":785596824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":785596824},{"type":"v1","addr":"172.21.15.188:6829","nonce":785596824}]},"public_addr":"172.21.15.188:6825/785596824","cluster_addr":"172.21.15.188:6827/785596824","heartbeat_back_addr":"172.21.15.188:6831/785596824","heartbeat_front_addr":"172.21.15.188:6829/785596824","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:31.754448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:54:55.094088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:18.861218+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:55:44.615295+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:09.573279+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:56:34.256476+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:00.269203+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:25.885538+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:57:51.675489+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:18.784844+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:58:44.732047+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:59:11.191759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/2804791885":"2024-04-25T23:52:54.426106+0000","172.21.15.110:6801/1818468115":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3421586240":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/3060209766":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6801/2433943234":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/2783100792":"2024-04-25T23:52:27.865188+0000","172.21.15.110:0/1429818736":"2024-04-25T23:52:54.426106+0000","172.21.15.110:0/1657096513":"2024-04-25T23:52:27.865188+0000","172.21.15.110:6800/1818468115":"2024-04-25T23:52:54.426106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:06.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fdc0a2c10 msgr2=0x7f5fdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fdc0a2c10 0x7f5fdc0a50d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5fec1016f0 tx=0x7f5fe000a2d0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 msgr2=0x7f5fec197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 0x7f5fec197f10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005940 tx=0x7f5fd8004430 comp rx=0 tx=0).stop 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 shutdown_connections 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f5fdc0a2c10 0x7f5fdc0a50d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fec102250 0x7f5fec19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101890 0x7f5fec197f10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 --2- 172.21.15.110:0/1234015088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fec1006a0 0x7f5fec1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 >> 172.21.15.110:0/1234015088 conn(0x7f5fec0fbc30 msgr2=0x7f5fec104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 shutdown_connections 2024-04-25T00:19:06.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.390+0000 7f5ff2b63640 1 -- 172.21.15.110:0/1234015088 wait complete. 2024-04-25T00:19:06.403 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:19:06.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.501+0000 7fafbfe8c640 1 -- 172.21.15.110:0/2328582590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 msgr2=0x7fafb8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.501+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/2328582590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 0x7fafb8101d10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009900 tx=0x7fafa802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 -- 172.21.15.110:0/2328582590 shutdown_connections 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/2328582590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafb8102250 0x7fafb8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/2328582590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 0x7fafb8101d10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/2328582590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafb81006a0 0x7fafb8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 -- 172.21.15.110:0/2328582590 >> 172.21.15.110:0/2328582590 conn(0x7fafb80fbc30 msgr2=0x7fafb80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 -- 172.21.15.110:0/2328582590 shutdown_connections 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 -- 172.21.15.110:0/2328582590 wait complete. 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 Processor -- start 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 -- start start 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafb81006a0 0x7fafb8196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 0x7fafb8196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafb8102250 0x7fafb819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810c1d0 con 0x7fafb8101890 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810c050 con 0x7fafb81006a0 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbfe8c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810c350 con 0x7fafb8102250 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbd427640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 0x7fafb8196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbd427640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 0x7fafb8196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43088/0 (socket says 172.21.15.110:43088) 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.502+0000 7fafbd427640 1 -- 172.21.15.110:0/1424821713 learned_addr learned my addr 172.21.15.110:0/1424821713 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbdc28640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafb81006a0 0x7fafb8196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbe429640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafb8102250 0x7fafb819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbd427640 1 -- 172.21.15.110:0/1424821713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafb8102250 msgr2=0x7fafb819dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbd427640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafb8102250 0x7fafb819dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbd427640 1 -- 172.21.15.110:0/1424821713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafb81006a0 msgr2=0x7fafb8196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbd427640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafb81006a0 0x7fafb8196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbd427640 1 -- 172.21.15.110:0/1424821713 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb819e530 con 0x7fafb8101890 2024-04-25T00:19:06.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbe429640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafb8102250 0x7fafb819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:06.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbd427640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 0x7fafb8196e50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fafa8005940 tx=0x7fafa8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:06.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafaeffd640 1 -- 172.21.15.110:0/1424821713 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa803f030 con 0x7fafb8101890 2024-04-25T00:19:06.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafaeffd640 1 -- 172.21.15.110:0/1424821713 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa8003cb0 con 0x7fafb8101890 2024-04-25T00:19:06.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb819e7c0 con 0x7fafb8101890 2024-04-25T00:19:06.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafaeffd640 1 -- 172.21.15.110:0/1424821713 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8035070 con 0x7fafb8101890 2024-04-25T00:19:06.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.503+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb819ecb0 con 0x7fafb8101890 2024-04-25T00:19:06.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.504+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb8107ae0 con 0x7fafb8101890 2024-04-25T00:19:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.505+0000 7fafaeffd640 1 -- 172.21.15.110:0/1424821713 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fafa8042030 con 0x7fafb8101890 2024-04-25T00:19:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.506+0000 7fafaeffd640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fafa40a2b90 0x7fafa40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.506+0000 7fafbdc28640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fafa40a2b90 0x7fafa40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.506+0000 7fafaeffd640 1 -- 172.21.15.110:0/1424821713 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8182+0+0 (secure 0 0 0) 0x7fafa80efdc0 con 0x7fafb8101890 2024-04-25T00:19:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.506+0000 7fafbdc28640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fafa40a2b90 0x7fafa40a5050 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fafb0005c40 tx=0x7fafb0008230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:06.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.509+0000 7fafaeffd640 1 -- 172.21.15.110:0/1424821713 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fafa80bcd50 con 0x7fafb8101890 2024-04-25T00:19:06.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.725+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fafb802d340 con 0x7fafb8101890 2024-04-25T00:19:06.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.728+0000 7fafaeffd640 1 -- 172.21.15.110:0/1424821713 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7fafa80bc650 con 0x7fafb8101890 2024-04-25T00:19:06.730 INFO:teuthology.orchestra.run.smithi110.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fafa40a2b90 msgr2=0x7fafa40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fafa40a2b90 0x7fafa40a5050 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fafb0005c40 tx=0x7fafb0008230 comp rx=0 tx=0).stop 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 msgr2=0x7fafb8196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 0x7fafb8196e50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fafa8005940 tx=0x7fafa8004430 comp rx=0 tx=0).stop 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 shutdown_connections 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fafa40a2b90 0x7fafa40a5050 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafb8102250 0x7fafb819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8101890 0x7fafb8196e50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 --2- 172.21.15.110:0/1424821713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafb81006a0 0x7fafb8196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 >> 172.21.15.110:0/1424821713 conn(0x7fafb80fbc30 msgr2=0x7fafb8104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:06.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 shutdown_connections 2024-04-25T00:19:06.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.730+0000 7fafbfe8c640 1 -- 172.21.15.110:0/1424821713 wait complete. 2024-04-25T00:19:06.743 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:19:06.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.845+0000 7fd9680a5640 1 -- 172.21.15.110:0/10072533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960100680 msgr2=0x7fd960100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.845+0000 7fd9680a5640 1 --2- 172.21.15.110:0/10072533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960100680 0x7fd960100aa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd954009900 tx=0x7fd95402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:06.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.845+0000 7fd9680a5640 1 -- 172.21.15.110:0/10072533 shutdown_connections 2024-04-25T00:19:06.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.845+0000 7fd9680a5640 1 --2- 172.21.15.110:0/10072533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd960102230 0x7fd960108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.845+0000 7fd9680a5640 1 --2- 172.21.15.110:0/10072533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd960101870 0x7fd960101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.845+0000 7fd9680a5640 1 --2- 172.21.15.110:0/10072533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960100680 0x7fd960100aa0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.845+0000 7fd9680a5640 1 -- 172.21.15.110:0/10072533 >> 172.21.15.110:0/10072533 conn(0x7fd9600fbbf0 msgr2=0x7fd9600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 -- 172.21.15.110:0/10072533 shutdown_connections 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 -- 172.21.15.110:0/10072533 wait complete. 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 Processor -- start 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 -- start start 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd960100680 0x7fd9601979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960101870 0x7fd960197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd960102230 0x7fd96019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd96010c180 con 0x7fd960101870 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd96010c000 con 0x7fd960100680 2024-04-25T00:19:06.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd9680a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd96010c300 con 0x7fd960102230 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd965640640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960101870 0x7fd960197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd965640640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960101870 0x7fd960197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43106/0 (socket says 172.21.15.110:43106) 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd965640640 1 -- 172.21.15.110:0/542197833 learned_addr learned my addr 172.21.15.110:0/542197833 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.846+0000 7fd966642640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd960102230 0x7fd96019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd965e41640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd960100680 0x7fd9601979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd965640640 1 -- 172.21.15.110:0/542197833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd960102230 msgr2=0x7fd96019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd965640640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd960102230 0x7fd96019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd965640640 1 -- 172.21.15.110:0/542197833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd960100680 msgr2=0x7fd9601979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd965640640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd960100680 0x7fd9601979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd965640640 1 -- 172.21.15.110:0/542197833 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96019f5c0 con 0x7fd960101870 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd965e41640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd960100680 0x7fd9601979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd965640640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960101870 0x7fd960197ee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00d950 tx=0x7fd95c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd952ffd640 1 -- 172.21.15.110:0/542197833 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c017070 con 0x7fd960101870 2024-04-25T00:19:06.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd952ffd640 1 -- 172.21.15.110:0/542197833 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd95c009c30 con 0x7fd960101870 2024-04-25T00:19:06.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96019f8b0 con 0x7fd960101870 2024-04-25T00:19:06.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd952ffd640 1 -- 172.21.15.110:0/542197833 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00f620 con 0x7fd960101870 2024-04-25T00:19:06.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.847+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96019fe50 con 0x7fd960101870 2024-04-25T00:19:06.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.848+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9601a0080 con 0x7fd960101870 2024-04-25T00:19:06.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.849+0000 7fd952ffd640 1 -- 172.21.15.110:0/542197833 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd95c0109a0 con 0x7fd960101870 2024-04-25T00:19:06.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.850+0000 7fd952ffd640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd94c0a2c10 0x7fd94c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.850+0000 7fd965e41640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd94c0a2c10 0x7fd94c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.850+0000 7fd952ffd640 1 -- 172.21.15.110:0/542197833 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8440+0+0 (secure 0 0 0) 0x7fd95c0cec80 con 0x7fd960101870 2024-04-25T00:19:06.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.850+0000 7fd965e41640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd94c0a2c10 0x7fd94c0a50d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd954000c00 tx=0x7fd954032d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:06.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:06.854+0000 7fd952ffd640 1 -- 172.21.15.110:0/542197833 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd95c009de0 con 0x7fd960101870 2024-04-25T00:19:07.036 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a[29183]: 2024-04-25T00:19:06.726+0000 7f12958a8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:19:07.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/4283849877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1234015088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1424821713' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:19:07.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:19:07.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:19:07.037 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/1424821713' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:19:07.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:19:07.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:19:07.038 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:06 smithi110 ceph-mon[29187]: pgmap v864: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-04-25T00:19:07.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.072+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd9601a0250 con 0x7fd960101870 2024-04-25T00:19:07.102 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: ignoring --setuser ceph since I am not root 2024-04-25T00:19:07.103 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: ignoring --setgroup ceph since I am not root 2024-04-25T00:19:07.103 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mds[62871]: -- 172.21.15.173:0/3387105699 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600dfb21a40 con 0x5600dfaf4800 2024-04-25T00:19:07.103 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: starting mds.b at 2024-04-25T00:19:07.103 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h[62559]: ignoring --setuser ceph since I am not root 2024-04-25T00:19:07.103 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h[62559]: ignoring --setgroup ceph since I am not root 2024-04-25T00:19:07.103 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mds[62563]: -- 172.21.15.173:0/3017565557 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617ed4b1a40 con 0x5617ed484c00 2024-04-25T00:19:07.103 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h[62559]: starting mds.h at 2024-04-25T00:19:07.103 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/4283849877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1234015088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1424821713' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/1424821713' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mon[32402]: pgmap v864: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e[62253]: ignoring --setuser ceph since I am not root 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e[62253]: ignoring --setgroup ceph since I am not root 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-mds[62257]: -- 172.21.15.173:0/1781296575 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559166017a40 con 0x559165feac00 2024-04-25T00:19:07.104 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:19:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e[62253]: starting mds.e at 2024-04-25T00:19:07.268 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c[63497]: ignoring --setuser ceph since I am not root 2024-04-25T00:19:07.268 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c[63497]: ignoring --setgroup ceph since I am not root 2024-04-25T00:19:07.268 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mds[63501]: -- 172.21.15.188:0/364526638 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a25be0ba40 con 0x55a25bddec00 2024-04-25T00:19:07.268 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c[63497]: starting mds.c at 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i[63805]: ignoring --setuser ceph since I am not root 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i[63805]: ignoring --setgroup ceph since I am not root 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mds[63809]: -- 172.21.15.188:0/2651989829 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c54245fa40 con 0x55c542433000 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i[63805]: starting mds.i at 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setuser ceph since I am not root 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setgroup ceph since I am not root 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mds[63193]: -- 172.21.15.188:0/2471892123 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622d2985a40 con 0x5622d2958c00 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f[63189]: starting mds.f at 2024-04-25T00:19:07.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/4283849877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1234015088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1424821713' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:19:07.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:19:07.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:19:07.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/1424821713' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:19:07.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:19:07.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:19:07.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:06 smithi188 ceph-mon[33285]: pgmap v864: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-04-25T00:19:07.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.733+0000 7fd952ffd640 1 -- 172.21.15.110:0/542197833 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7fd95c09bb90 con 0x7fd960101870 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd94c0a2c10 msgr2=0x7fd94c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd94c0a2c10 0x7fd94c0a50d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd954000c00 tx=0x7fd954032d20 comp rx=0 tx=0).stop 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960101870 msgr2=0x7fd960197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960101870 0x7fd960197ee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00d950 tx=0x7fd95c00de10 comp rx=0 tx=0).stop 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 shutdown_connections 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7fd94c0a2c10 0x7fd94c0a50d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd960102230 0x7fd96019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd960101870 0x7fd960197ee0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 --2- 172.21.15.110:0/542197833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd960100680 0x7fd9601979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 >> 172.21.15.110:0/542197833 conn(0x7fd9600fbbf0 msgr2=0x7fd960068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 shutdown_connections 2024-04-25T00:19:07.737 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.736+0000 7fd9680a5640 1 -- 172.21.15.110:0/542197833 wait complete. 2024-04-25T00:19:07.751 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:19:07.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 -- 172.21.15.110:0/728656981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 msgr2=0x7ff5a81045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:07.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/728656981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 0x7ff5a81045f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff598009900 tx=0x7ff59802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 -- 172.21.15.110:0/728656981 shutdown_connections 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/728656981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5a8107460 0x7ff5a81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/728656981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5a8104b30 0x7ff5a8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/728656981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 0x7ff5a81045f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 -- 172.21.15.110:0/728656981 >> 172.21.15.110:0/728656981 conn(0x7ff5a80fbbf0 msgr2=0x7ff5a80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 -- 172.21.15.110:0/728656981 shutdown_connections 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 -- 172.21.15.110:0/728656981 wait complete. 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 Processor -- start 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.858+0000 7ff5acdf9640 1 -- start start 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5acdf9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 0x7ff5a8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5acdf9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5a8104b30 0x7ff5a8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5acdf9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5a8107460 0x7ff5a819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:07.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5acdf9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a810d320 con 0x7ff5a8102200 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5acdf9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a810d1a0 con 0x7ff5a8107460 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5acdf9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a810d4a0 con 0x7ff5a8104b30 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 0x7ff5a8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 0x7ff5a8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43124/0 (socket says 172.21.15.110:43124) 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 -- 172.21.15.110:0/2405479323 learned_addr learned my addr 172.21.15.110:0/2405479323 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff59ffff640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5a8104b30 0x7ff5a8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a6d9d640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5a8107460 0x7ff5a819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 -- 172.21.15.110:0/2405479323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5a8104b30 msgr2=0x7ff5a8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5a8104b30 0x7ff5a8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 -- 172.21.15.110:0/2405479323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5a8107460 msgr2=0x7ff5a819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5a8107460 0x7ff5a819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 -- 172.21.15.110:0/2405479323 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a819f670 con 0x7ff5a8102200 2024-04-25T00:19:07.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5a659c640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 0x7ff5a8197a50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff5980314e0 tx=0x7ff598003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff59f7fe640 1 -- 172.21.15.110:0/2405479323 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59803f070 con 0x7ff5a8102200 2024-04-25T00:19:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff59f7fe640 1 -- 172.21.15.110:0/2405479323 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff598031e40 con 0x7ff5a8102200 2024-04-25T00:19:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a819f900 con 0x7ff5a8102200 2024-04-25T00:19:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.859+0000 7ff59f7fe640 1 -- 172.21.15.110:0/2405479323 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598035030 con 0x7ff5a8102200 2024-04-25T00:19:07.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.860+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a819fdf0 con 0x7ff5a8102200 2024-04-25T00:19:07.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.860+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a80fd6e0 con 0x7ff5a8102200 2024-04-25T00:19:07.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.861+0000 7ff59f7fe640 1 -- 172.21.15.110:0/2405479323 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff598042070 con 0x7ff5a8102200 2024-04-25T00:19:07.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.862+0000 7ff59f7fe640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff5940a2b90 0x7ff5940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:07.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.862+0000 7ff59f7fe640 1 -- 172.21.15.110:0/2405479323 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8440+0+0 (secure 0 0 0) 0x7ff5980ef9b0 con 0x7ff5a8102200 2024-04-25T00:19:07.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.862+0000 7ff59ffff640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff5940a2b90 0x7ff5940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:07.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.863+0000 7ff59ffff640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff5940a2b90 0x7ff5940a5050 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8198f60 tx=0x7ff59000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:07.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:07.867+0000 7ff59f7fe640 1 -- 172.21.15.110:0/2405479323 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff5980bc1f0 con 0x7ff5a8102200 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/542197833' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: Cluster is now healthy 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/542197833' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6834/4126885794,v1:172.21.15.173:6835/4126885794] up:boot 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6836/876147494,v1:172.21.15.173:6837/876147494] up:boot 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.173:6838/1577782584,v1:172.21.15.173:6839/1577782584] up:boot 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.188:6834/971318020,v1:172.21.15.188:6837/971318020] up:boot 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.188:6835/3074476208,v1:172.21.15.188:6838/3074476208] up:boot 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: mds.? [v2:172.21.15.188:6836/2002915282,v1:172.21.15.188:6839/2002915282] up:boot 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: fsmap 12 up:standby 2024-04-25T00:19:08.025 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:19:08.026 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:19:08.026 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:19:08.026 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:19:08.026 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:19:08.026 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:07 smithi110 ceph-mon[29187]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:19:08.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.053+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff5a8062340 con 0x7ff5a8102200 2024-04-25T00:19:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/542197833' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:19:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:19:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:19:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: Cluster is now healthy 2024-04-25T00:19:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/542197833' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:19:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6834/4126885794,v1:172.21.15.173:6835/4126885794] up:boot 2024-04-25T00:19:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6836/876147494,v1:172.21.15.173:6837/876147494] up:boot 2024-04-25T00:19:08.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.173:6838/1577782584,v1:172.21.15.173:6839/1577782584] up:boot 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.188:6834/971318020,v1:172.21.15.188:6837/971318020] up:boot 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.188:6835/3074476208,v1:172.21.15.188:6838/3074476208] up:boot 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: mds.? [v2:172.21.15.188:6836/2002915282,v1:172.21.15.188:6839/2002915282] up:boot 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: fsmap 12 up:standby 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:19:08.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:07 smithi188 ceph-mon[33285]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:19:08.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/542197833' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: Cluster is now healthy 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/542197833' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6834/4126885794,v1:172.21.15.173:6835/4126885794] up:boot 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6836/876147494,v1:172.21.15.173:6837/876147494] up:boot 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.173:6838/1577782584,v1:172.21.15.173:6839/1577782584] up:boot 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.188:6834/971318020,v1:172.21.15.188:6837/971318020] up:boot 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.188:6835/3074476208,v1:172.21.15.188:6838/3074476208] up:boot 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: mds.? [v2:172.21.15.188:6836/2002915282,v1:172.21.15.188:6839/2002915282] up:boot 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: fsmap 12 up:standby 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:19:08.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:19:08.354 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:19:08.354 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:19:08.354 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:19:08.354 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:07 smithi173 ceph-mon[32402]: from='mgr.14152 172.21.15.110:0/4039693909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:19:08.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.913+0000 7ff59f7fe640 1 -- 172.21.15.110:0/2405479323 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7ff5980c04f0 con 0x7ff5a8102200 2024-04-25T00:19:08.914 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:19:08.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff5940a2b90 msgr2=0x7ff5940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:08.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff5940a2b90 0x7ff5940a5050 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8198f60 tx=0x7ff59000b410 comp rx=0 tx=0).stop 2024-04-25T00:19:08.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 msgr2=0x7ff5a8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:08.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 0x7ff5a8197a50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff5980314e0 tx=0x7ff598003c80 comp rx=0 tx=0).stop 2024-04-25T00:19:08.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 shutdown_connections 2024-04-25T00:19:08.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7ff5940a2b90 0x7ff5940a5050 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5a8107460 0x7ff5a819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5a8104b30 0x7ff5a8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 --2- 172.21.15.110:0/2405479323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5a8102200 0x7ff5a8197a50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 >> 172.21.15.110:0/2405479323 conn(0x7ff5a80fbbf0 msgr2=0x7ff5a81040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:08.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.916+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 shutdown_connections 2024-04-25T00:19:08.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:08.917+0000 7ff5acdf9640 1 -- 172.21.15.110:0/2405479323 wait complete. 2024-04-25T00:19:08.930 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:19:09.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 -- 172.21.15.110:0/2726854353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24100690 msgr2=0x7f3b24100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:09.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 --2- 172.21.15.110:0/2726854353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24100690 0x7f3b24100ab0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009900 tx=0x7f3b1402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:09.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 -- 172.21.15.110:0/2726854353 shutdown_connections 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 --2- 172.21.15.110:0/2726854353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b24102240 0x7f3b24108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 --2- 172.21.15.110:0/2726854353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24101880 0x7f3b24101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 --2- 172.21.15.110:0/2726854353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24100690 0x7f3b24100ab0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 -- 172.21.15.110:0/2726854353 >> 172.21.15.110:0/2726854353 conn(0x7f3b240fbc00 msgr2=0x7f3b240fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 -- 172.21.15.110:0/2726854353 shutdown_connections 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.043+0000 7f3b29a15640 1 -- 172.21.15.110:0/2726854353 wait complete. 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b29a15640 1 Processor -- start 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b29a15640 1 -- start start 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b29a15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24100690 0x7f3b241045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b29a15640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24101880 0x7f3b24102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b29a15640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b24102240 0x7f3b241031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:09.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b29a15640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410c290 con 0x7f3b24101880 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b29a15640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410c110 con 0x7f3b24102240 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b29a15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410c410 con 0x7f3b24100690 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24101880 0x7f3b24102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24101880 0x7f3b24102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43148/0 (socket says 172.21.15.110:43148) 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 -- 172.21.15.110:0/3577354129 learned_addr learned my addr 172.21.15.110:0/3577354129 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b237fe640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b24102240 0x7f3b241031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 -- 172.21.15.110:0/3577354129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24100690 msgr2=0x7f3b241045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b22ffd640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24100690 0x7f3b241045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24100690 0x7f3b241045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 -- 172.21.15.110:0/3577354129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b24102240 msgr2=0x7f3b241031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b24102240 0x7f3b241031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 -- 172.21.15.110:0/3577354129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b24103a60 con 0x7f3b24101880 2024-04-25T00:19:09.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b237fe640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b24102240 0x7f3b241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b1bfff640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24101880 0x7f3b24102c60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00b4a0 tx=0x7f3b0c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.044+0000 7f3b28a13640 1 -- 172.21.15.110:0/3577354129 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c014070 con 0x7f3b24101880 2024-04-25T00:19:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.045+0000 7f3b28a13640 1 -- 172.21.15.110:0/3577354129 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b0c010ac0 con 0x7f3b24101880 2024-04-25T00:19:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.045+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b241a1d60 con 0x7f3b24101880 2024-04-25T00:19:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.045+0000 7f3b28a13640 1 -- 172.21.15.110:0/3577354129 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c017c30 con 0x7f3b24101880 2024-04-25T00:19:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.045+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b241a2250 con 0x7f3b24101880 2024-04-25T00:19:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.046+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b241a24e0 con 0x7f3b24101880 2024-04-25T00:19:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.047+0000 7f3b28a13640 1 -- 172.21.15.110:0/3577354129 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3b0c017070 con 0x7f3b24101880 2024-04-25T00:19:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.047+0000 7f3b28a13640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b100a2b90 0x7f3b100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.047+0000 7f3b22ffd640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b100a2b90 0x7f3b100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.047+0000 7f3b28a13640 1 -- 172.21.15.110:0/3577354129 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7886+0+0 (secure 0 0 0) 0x7f3b0c0ce2f0 con 0x7f3b24101880 2024-04-25T00:19:09.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.048+0000 7f3b22ffd640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b100a2b90 0x7f3b100a5050 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b14006010 tx=0x7f3b14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:09.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.052+0000 7f3b28a13640 1 -- 172.21.15.110:0/3577354129 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b0c09b400 con 0x7f3b24101880 2024-04-25T00:19:09.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.234+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3b241a26b0 con 0x7f3b24101880 2024-04-25T00:19:09.254 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:08 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2405479323' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:19:09.254 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:08 smithi173 ceph-mon[32402]: pgmap v865: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-04-25T00:19:09.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:08 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2405479323' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:19:09.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:08 smithi188 ceph-mon[33285]: pgmap v865: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-04-25T00:19:09.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:08 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2405479323' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:19:09.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:08 smithi110 ceph-mon[29187]: pgmap v865: 97 pgs: 97 active+clean; 1.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-04-25T00:19:09.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.915+0000 7f3b28a13640 1 -- 172.21.15.110:0/3577354129 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7f3b0c09ad00 con 0x7f3b24101880 2024-04-25T00:19:09.915 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephfs_data' removed 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b100a2b90 msgr2=0x7f3b100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b100a2b90 0x7f3b100a5050 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b14006010 tx=0x7f3b14000bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24101880 msgr2=0x7f3b24102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24101880 0x7f3b24102c60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00b4a0 tx=0x7f3b0c00b960 comp rx=0 tx=0).stop 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 shutdown_connections 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:6800/546485163,v1:172.21.15.110:6801/546485163] conn(0x7f3b100a2b90 0x7f3b100a5050 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b24102240 0x7f3b241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b24101880 0x7f3b24102c60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 --2- 172.21.15.110:0/3577354129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24100690 0x7f3b241045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 >> 172.21.15.110:0/3577354129 conn(0x7f3b240fbc00 msgr2=0x7f3b24069a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:09.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 shutdown_connections 2024-04-25T00:19:09.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T00:19:09.918+0000 7f3b29a15640 1 -- 172.21.15.110:0/3577354129 wait complete. 2024-04-25T00:19:09.938 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:19:09.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:19:10.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:19:10.039 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T00:19:10.039 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:19:10.088 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:19:10.112 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:19:10.128 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T00:19:10.128 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T00:19:10.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:19:10.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:09 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/2405479323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:19:10.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:09 smithi188 ceph-mon[33285]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:19:10.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:09 smithi188 ceph-mon[33285]: from='client.? 172.21.15.110:0/3577354129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:19:10.286 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:09 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/2405479323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:19:10.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:09 smithi110 ceph-mon[29187]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:19:10.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:09 smithi110 ceph-mon[29187]: from='client.? 172.21.15.110:0/3577354129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:19:10.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:10 smithi110 systemd[1]: Stopping Ceph mon.a for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:10.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:09 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/2405479323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:19:10.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:09 smithi173 ceph-mon[32402]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:19:10.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:09 smithi173 ceph-mon[32402]: from='client.? 172.21.15.110:0/3577354129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:19:10.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a[29183]: 2024-04-25T00:19:10.476+0000 7f129b0b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:10.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a[29183]: 2024-04-25T00:19:10.476+0000 7f129b0b3700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T00:19:10.787 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:10 smithi110 podman[93495]: 2024-04-25 00:19:10.671261114 +0000 UTC m=+0.343890682 container died 0ca456db64e894722112587243b9548b4f4369f2fd84b188dffa298ae2b74521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T00:19:11.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:11 smithi110 podman[93495]: 2024-04-25 00:19:11.052703859 +0000 UTC m=+0.725333422 container cleanup 0ca456db64e894722112587243b9548b4f4369f2fd84b188dffa298ae2b74521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:19:11.287 INFO:journalctl@ceph.mon.a.smithi110.stdout:Apr 25 00:19:11 smithi110 bash[93495]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:19:11.582 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T00:19:11.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:11.632 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T00:19:11.632 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T00:19:11.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.b 2024-04-25T00:19:11.984 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:11 smithi173 systemd[1]: Stopping Ceph mon.b for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:12.352 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:11 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-b[32398]: 2024-04-25T00:19:11.983+0000 7efcf1eaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:12.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:11 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-b[32398]: 2024-04-25T00:19:11.983+0000 7efcf1eaf700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:19:12.353 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:12 smithi173 podman[68377]: 2024-04-25 00:19:12.126731343 +0000 UTC m=+0.272192086 container died b9c86dc5242f54745c99e09c06acd64e5ec734e7e4c988f5cd5addd16e1222ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:19:12.852 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:12 smithi173 podman[68377]: 2024-04-25 00:19:12.592682684 +0000 UTC m=+0.738143418 container cleanup b9c86dc5242f54745c99e09c06acd64e5ec734e7e4c988f5cd5addd16e1222ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T00:19:12.853 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 25 00:19:12 smithi173 bash[68377]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-b 2024-04-25T00:19:13.059 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T00:19:13.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:13.103 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T00:19:13.103 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T00:19:13.103 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.c 2024-04-25T00:19:13.489 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:13 smithi188 systemd[1]: Stopping Ceph mon.c for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c[33267]: 2024-04-25T00:19:13.488+0000 7ff81f25c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c[33267]: 2024-04-25T00:19:13.488+0000 7ff81f25c700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:19:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:13 smithi188 podman[69602]: 2024-04-25 00:19:13.694021191 +0000 UTC m=+0.328769663 container died c462243bdd476cc2be0ef2813924cdfc06b015b89f38503519b70cab25c80685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T00:19:14.474 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:14 smithi188 podman[69602]: 2024-04-25 00:19:14.390479847 +0000 UTC m=+1.025228305 container cleanup c462243bdd476cc2be0ef2813924cdfc06b015b89f38503519b70cab25c80685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T00:19:14.474 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:14 smithi188 bash[69602]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c 2024-04-25T00:19:14.763 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 25 00:19:14 smithi188 podman[69616]: 2024-04-25 00:19:14.634390968 +0000 UTC m=+0.936305755 container remove c462243bdd476cc2be0ef2813924cdfc06b015b89f38503519b70cab25c80685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T00:19:14.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T00:19:14.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:14.831 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T00:19:14.831 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T00:19:14.831 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:19:15.140 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:19:14 smithi110 systemd[1]: Stopping Ceph mgr.x for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:15.475 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:19:15 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-25T00:19:15.140+0000 7f51d967d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:15.475 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:19:15 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x[29419]: 2024-04-25T00:19:15.140+0000 7f51d967d700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T00:19:15.475 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:19:15 smithi110 podman[93617]: 2024-04-25 00:19:15.17130038 +0000 UTC m=+0.154139517 container died 6592987f0c90f8aae6e8f29bfa5ba2705cc35e0d455928314b48ad2bf3348121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T00:19:15.786 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:19:15 smithi110 podman[93617]: 2024-04-25 00:19:15.583452101 +0000 UTC m=+0.566291240 container cleanup 6592987f0c90f8aae6e8f29bfa5ba2705cc35e0d455928314b48ad2bf3348121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:19:15.786 INFO:journalctl@ceph.mgr.x.smithi110.stdout:Apr 25 00:19:15 smithi110 bash[93617]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-x 2024-04-25T00:19:15.928 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T00:19:15.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:15.964 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T00:19:15.964 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T00:19:15.964 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.y 2024-04-25T00:19:16.303 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 25 00:19:16 smithi173 systemd[1]: Stopping Ceph mgr.y for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:16.594 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 25 00:19:16 smithi173 podman[68503]: 2024-04-25 00:19:16.303667927 +0000 UTC m=+0.143432967 container died ab3af38ff9f5b9f1126491f0567098657b46e1c3345896e87cfa2b274074c125 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T00:19:16.852 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 25 00:19:16 smithi173 podman[68503]: 2024-04-25 00:19:16.684692178 +0000 UTC m=+0.524457233 container cleanup ab3af38ff9f5b9f1126491f0567098657b46e1c3345896e87cfa2b274074c125 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:19:16.852 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 25 00:19:16 smithi173 bash[68503]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y 2024-04-25T00:19:17.160 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 25 00:19:16 smithi173 podman[68517]: 2024-04-25 00:19:16.903549645 +0000 UTC m=+0.595699320 container remove ab3af38ff9f5b9f1126491f0567098657b46e1c3345896e87cfa2b274074c125 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:19:17.161 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 25 00:19:16 smithi173 systemd[1]: ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T00:19:17.164 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T00:19:17.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:17.200 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T00:19:17.200 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T00:19:17.200 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.z 2024-04-25T00:19:17.518 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:17 smithi188 systemd[1]: Stopping Ceph mgr.z for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:17.965 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:17 smithi188 podman[69736]: 2024-04-25 00:19:17.641809674 +0000 UTC m=+0.246557941 container died 7ac6894c5fa2d77003235b052ca6bb0b698fd8428179132f3b4ce1d643d90469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T00:19:18.268 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:18 smithi188 podman[69736]: 2024-04-25 00:19:18.063420471 +0000 UTC m=+0.668168734 container cleanup 7ac6894c5fa2d77003235b052ca6bb0b698fd8428179132f3b4ce1d643d90469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:19:18.268 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:18 smithi188 bash[69736]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z 2024-04-25T00:19:18.529 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T00:19:18.549 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:18 smithi188 podman[69750]: 2024-04-25 00:19:18.290136857 +0000 UTC m=+0.630078910 container remove 7ac6894c5fa2d77003235b052ca6bb0b698fd8428179132f3b4ce1d643d90469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T00:19:18.549 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:18 smithi188 systemd[1]: ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-25T00:19:18.549 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:18 smithi188 systemd[1]: ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-04-25T00:19:18.549 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:18 smithi188 systemd[1]: Stopped Ceph mgr.z for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-25T00:19:18.549 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 25 00:19:18 smithi188 systemd[1]: ceph-787fa75a-0295-11ef-bc93-c7b262605968@mgr.z.service: Consumed 14.340s CPU time. 2024-04-25T00:19:18.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:18.564 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T00:19:18.564 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T00:19:18.564 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.0 2024-04-25T00:19:18.786 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:18 smithi110 systemd[1]: Stopping Ceph osd.0 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:19.286 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:18 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0[44124]: 2024-04-25T00:19:18.896+0000 7f8fc357d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:19.287 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:18 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0[44124]: 2024-04-25T00:19:18.896+0000 7f8fc357d700 -1 osd.0 90 *** Got signal Terminated *** 2024-04-25T00:19:19.287 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:18 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0[44124]: 2024-04-25T00:19:18.896+0000 7f8fc357d700 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:19:24.744 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:24 smithi110 podman[93741]: 2024-04-25 00:19:24.438840641 +0000 UTC m=+5.690125078 container died 27937edf1e3a63b3e49f02eba2a3d6a0faacb66f0b33210bf4ebd9b4f94a2fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T00:19:25.036 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:24 smithi110 podman[93741]: 2024-04-25 00:19:24.855921256 +0000 UTC m=+6.107205700 container cleanup 27937edf1e3a63b3e49f02eba2a3d6a0faacb66f0b33210bf4ebd9b4f94a2fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T00:19:25.036 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:24 smithi110 bash[93741]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0 2024-04-25T00:19:25.288 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:25 smithi110 podman[93758]: 2024-04-25 00:19:25.132702082 +0000 UTC m=+0.681611262 container remove 27937edf1e3a63b3e49f02eba2a3d6a0faacb66f0b33210bf4ebd9b4f94a2fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T00:19:25.544 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:25 smithi110 podman[93850]: 2024-04-25 00:19:25.388467218 +0000 UTC m=+0.018037996 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:25.820 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:25 smithi110 podman[93850]: 2024-04-25 00:19:25.544562939 +0000 UTC m=+0.174133697 container create 2b481dc1629dad706f8b0a12bdf78ee023cb1088417942713dcdfe9c140c4fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T00:19:26.286 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:25 smithi110 podman[93850]: 2024-04-25 00:19:25.820333504 +0000 UTC m=+0.449904264 container init 2b481dc1629dad706f8b0a12bdf78ee023cb1088417942713dcdfe9c140c4fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:19:26.287 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:25 smithi110 podman[93850]: 2024-04-25 00:19:25.824198554 +0000 UTC m=+0.453769314 container start 2b481dc1629dad706f8b0a12bdf78ee023cb1088417942713dcdfe9c140c4fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:19:26.287 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:25 smithi110 podman[93850]: 2024-04-25 00:19:25.953828792 +0000 UTC m=+0.583399554 container attach 2b481dc1629dad706f8b0a12bdf78ee023cb1088417942713dcdfe9c140c4fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:19:26.641 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 00:19:26 smithi110 podman[93850]: 2024-04-25 00:19:26.36974612 +0000 UTC m=+0.999316871 container died 2b481dc1629dad706f8b0a12bdf78ee023cb1088417942713dcdfe9c140c4fa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T00:19:27.001 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T00:19:27.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:27.041 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T00:19:27.041 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T00:19:27.041 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.1 2024-04-25T00:19:27.352 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:27 smithi173 systemd[1]: Stopping Ceph osd.1 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:28.102 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1[37000]: 2024-04-25T00:19:27.606+0000 7f76a94a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:28.103 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1[37000]: 2024-04-25T00:19:27.606+0000 7f76a94a4700 -1 osd.1 90 *** Got signal Terminated *** 2024-04-25T00:19:28.103 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1[37000]: 2024-04-25T00:19:27.606+0000 7f76a94a4700 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:19:33.501 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:33 smithi173 podman[68628]: 2024-04-25 00:19:33.163631614 +0000 UTC m=+5.934424328 container died f823e647956b6d7389933c7e52f7f1e5eb5f7669c70e8a2528ce580e5e0945d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T00:19:33.852 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:33 smithi173 podman[68628]: 2024-04-25 00:19:33.624966862 +0000 UTC m=+6.395759585 container cleanup f823e647956b6d7389933c7e52f7f1e5eb5f7669c70e8a2528ce580e5e0945d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T00:19:33.852 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:33 smithi173 bash[68628]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1 2024-04-25T00:19:34.153 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:33 smithi173 podman[68645]: 2024-04-25 00:19:33.859922326 +0000 UTC m=+0.635772049 container remove f823e647956b6d7389933c7e52f7f1e5eb5f7669c70e8a2528ce580e5e0945d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T00:19:34.422 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:34 smithi173 podman[68739]: 2024-04-25 00:19:34.056691811 +0000 UTC m=+0.017962072 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:34.423 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:34 smithi173 podman[68739]: 2024-04-25 00:19:34.177672886 +0000 UTC m=+0.138943134 container create 9addda0844d9fa46a8e42fa2a7f6d169002781575a869e9bae161c8becb4261d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T00:19:34.758 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:34 smithi173 podman[68739]: 2024-04-25 00:19:34.423035267 +0000 UTC m=+0.384305519 container init 9addda0844d9fa46a8e42fa2a7f6d169002781575a869e9bae161c8becb4261d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T00:19:34.759 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:34 smithi173 podman[68739]: 2024-04-25 00:19:34.426519585 +0000 UTC m=+0.387789829 container start 9addda0844d9fa46a8e42fa2a7f6d169002781575a869e9bae161c8becb4261d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T00:19:34.759 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:34 smithi173 podman[68739]: 2024-04-25 00:19:34.489688035 +0000 UTC m=+0.450958301 container attach 9addda0844d9fa46a8e42fa2a7f6d169002781575a869e9bae161c8becb4261d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:19:35.068 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 25 00:19:34 smithi173 podman[68739]: 2024-04-25 00:19:34.759292109 +0000 UTC m=+0.720562359 container died 9addda0844d9fa46a8e42fa2a7f6d169002781575a869e9bae161c8becb4261d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:19:35.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T00:19:35.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:35.435 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T00:19:35.435 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T00:19:35.435 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.2 2024-04-25T00:19:35.751 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:35 smithi188 systemd[1]: Stopping Ceph osd.2 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:36.018 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2[38443]: 2024-04-25T00:19:35.750+0000 7fcc113ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:36.018 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2[38443]: 2024-04-25T00:19:35.750+0000 7fcc113ca700 -1 osd.2 90 *** Got signal Terminated *** 2024-04-25T00:19:36.018 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2[38443]: 2024-04-25T00:19:35.750+0000 7fcc113ca700 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:19:41.485 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:41 smithi188 podman[69861]: 2024-04-25 00:19:41.216735485 +0000 UTC m=+5.587792593 container died 53baacc260e0b72971f2ed822d54ab437dafc9fde7272fb5ae71c4af2b2c2f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-25T00:19:41.768 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:41 smithi188 podman[69861]: 2024-04-25 00:19:41.576116026 +0000 UTC m=+5.947173132 container cleanup 53baacc260e0b72971f2ed822d54ab437dafc9fde7272fb5ae71c4af2b2c2f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:19:41.768 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:41 smithi188 bash[69861]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2 2024-04-25T00:19:42.197 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:41 smithi188 podman[69878]: 2024-04-25 00:19:41.861821714 +0000 UTC m=+0.642596512 container remove 53baacc260e0b72971f2ed822d54ab437dafc9fde7272fb5ae71c4af2b2c2f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:19:42.458 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:42 smithi188 podman[69970]: 2024-04-25 00:19:42.100785566 +0000 UTC m=+0.017707590 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:42.459 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:42 smithi188 podman[69970]: 2024-04-25 00:19:42.222063599 +0000 UTC m=+0.138985638 container create 0733dac59db8aa8506ae6b758b15e371eca6d570d57d64cdcaef36c6a2103828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T00:19:42.768 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:42 smithi188 podman[69970]: 2024-04-25 00:19:42.458666642 +0000 UTC m=+0.375588644 container init 0733dac59db8aa8506ae6b758b15e371eca6d570d57d64cdcaef36c6a2103828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:19:42.768 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:42 smithi188 podman[69970]: 2024-04-25 00:19:42.462306549 +0000 UTC m=+0.379228553 container start 0733dac59db8aa8506ae6b758b15e371eca6d570d57d64cdcaef36c6a2103828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T00:19:42.768 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:42 smithi188 podman[69970]: 2024-04-25 00:19:42.583907168 +0000 UTC m=+0.500829175 container attach 0733dac59db8aa8506ae6b758b15e371eca6d570d57d64cdcaef36c6a2103828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T00:19:43.268 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 25 00:19:42 smithi188 podman[69970]: 2024-04-25 00:19:42.954956309 +0000 UTC m=+0.871878322 container died 0733dac59db8aa8506ae6b758b15e371eca6d570d57d64cdcaef36c6a2103828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T00:19:43.596 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T00:19:43.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:43.635 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T00:19:43.636 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T00:19:43.636 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.3 2024-04-25T00:19:43.986 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:43 smithi110 systemd[1]: Stopping Ceph osd.3 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:44.286 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:43 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3[50643]: 2024-04-25T00:19:43.985+0000 7fb5d0a2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:44.287 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:43 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3[50643]: 2024-04-25T00:19:43.985+0000 7fb5d0a2b700 -1 osd.3 90 *** Got signal Terminated *** 2024-04-25T00:19:44.287 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:43 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3[50643]: 2024-04-25T00:19:43.985+0000 7fb5d0a2b700 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:19:46.036 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:45 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:45.700+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:46.709 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:46 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3[50643]: 2024-04-25T00:19:46.358+0000 7fb5c9017700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.739340+0000 front 2024-04-25T00:19:22.739310+0000 (oldest deadline 2024-04-25T00:19:45.639478+0000) 2024-04-25T00:19:47.036 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:46 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:46.708+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:47.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:47.219+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:47.519 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:47.451+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:47.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:47 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:47.319+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:47.671 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:47 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3[50643]: 2024-04-25T00:19:47.349+0000 7fb5c9017700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.739340+0000 front 2024-04-25T00:19:22.739310+0000 (oldest deadline 2024-04-25T00:19:45.639478+0000) 2024-04-25T00:19:48.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:47.553+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:48.036 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:47 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:47.906+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:48.036 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:47 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:47.670+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:48.470 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:48.207+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:48.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:48 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:48.307+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:48.720 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:48 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3[50643]: 2024-04-25T00:19:48.375+0000 7fb5c9017700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.739340+0000 front 2024-04-25T00:19:22.739310+0000 (oldest deadline 2024-04-25T00:19:45.639478+0000) 2024-04-25T00:19:48.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:48.545+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:48.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:48.470+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:49.036 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:48 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:48.720+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:49.037 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:48 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:48.869+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:49.458 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:49.161+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:49.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:49 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:49.296+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:49.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:49.546+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:49.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:49.458+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:49.882 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:49 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:49.732+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:49.883 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:49 smithi110 podman[93968]: 2024-04-25 00:19:49.56335931 +0000 UTC m=+5.733958990 container died 54ba0f9ea16a0850ea7af7f0d61a8f70f8f2b4d004efc615274aa0c83ad019d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:19:50.231 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:49 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:49.882+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:50.231 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:49 smithi110 podman[93968]: 2024-04-25 00:19:49.978271949 +0000 UTC m=+6.148871664 container cleanup 54ba0f9ea16a0850ea7af7f0d61a8f70f8f2b4d004efc615274aa0c83ad019d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:19:50.232 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:49 smithi110 bash[93968]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3 2024-04-25T00:19:50.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:50.434+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:50.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:50.188+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:50.532 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:50 smithi110 podman[93984]: 2024-04-25 00:19:50.231834631 +0000 UTC m=+0.661575737 container remove 54ba0f9ea16a0850ea7af7f0d61a8f70f8f2b4d004efc615274aa0c83ad019d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T00:19:50.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:50 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:50.337+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:50.786 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:50 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:50.762+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:50.787 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:50 smithi110 podman[94075]: 2024-04-25 00:19:50.43516973 +0000 UTC m=+0.018155128 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:50.787 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:50 smithi110 podman[94075]: 2024-04-25 00:19:50.541079692 +0000 UTC m=+0.124065074 container create cef3f2cce22bf721c4374ec6674e045b7c1eeb00cdb0dc74525dbca50a1a9e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T00:19:50.787 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:50 smithi110 podman[94075]: 2024-04-25 00:19:50.776798665 +0000 UTC m=+0.359784047 container init cef3f2cce22bf721c4374ec6674e045b7c1eeb00cdb0dc74525dbca50a1a9e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-25T00:19:50.787 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:50 smithi110 podman[94075]: 2024-04-25 00:19:50.77982318 +0000 UTC m=+0.362808562 container start cef3f2cce22bf721c4374ec6674e045b7c1eeb00cdb0dc74525dbca50a1a9e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:19:51.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:50.546+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:51.063 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:50 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:50.878+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:51.063 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:50 smithi110 podman[94075]: 2024-04-25 00:19:50.893715306 +0000 UTC m=+0.476700692 container attach cef3f2cce22bf721c4374ec6674e045b7c1eeb00cdb0dc74525dbca50a1a9e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:19:51.336 INFO:journalctl@ceph.osd.3.smithi110.stdout:Apr 25 00:19:51 smithi110 podman[94075]: 2024-04-25 00:19:51.063315105 +0000 UTC m=+0.646300488 container died cef3f2cce22bf721c4374ec6674e045b7c1eeb00cdb0dc74525dbca50a1a9e39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-25T00:19:51.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:51.219+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:51.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:51.409+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:51.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:51 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:51.308+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:51.760 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T00:19:51.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:51.798 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T00:19:51.798 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T00:19:51.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.4 2024-04-25T00:19:52.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:51.524+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:52.036 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:51 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:51.794+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:52.036 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:51 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:51.921+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:52.098 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:51 smithi173 systemd[1]: Stopping Ceph osd.4 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:19:52.352 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:52 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:52.325+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:52.352 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:52 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4[43925]: 2024-04-25T00:19:52.097+0000 7fe4bc316700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:19:52.352 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:52 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4[43925]: 2024-04-25T00:19:52.097+0000 7fe4bc316700 -1 osd.4 90 *** Got signal Terminated *** 2024-04-25T00:19:52.353 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:52 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4[43925]: 2024-04-25T00:19:52.097+0000 7fe4bc316700 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:19:52.500 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:52 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:52.206+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:52.501 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:52 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:52.435+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:52.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:52 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:52.500+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:53.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:52 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:52.794+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:53.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:52 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:52.961+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:53.518 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:53 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:53.488+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:53.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:53 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:53.401+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:53.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:53 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:53.245+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:53.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:53 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:53.358+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:54.286 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:53 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:53.946+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:54.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:53 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:53.810+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:54.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:53 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:53.810+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:19:54.498 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:54 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:54.244+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:54.499 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:54 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:54.361+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:54.499 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:54 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:54.361+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:19:54.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:54 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:54.342+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:54.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:54 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:54.342+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:19:54.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:19:54 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:19:54.446+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:19:54.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:54 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:54.497+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:55.188 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:54 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:54.965+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:55.188 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:54 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:54.843+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:55.188 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:54 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:54.843+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:19:55.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:55 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:55.201+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:55.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:55 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:55.201+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:19:55.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:55 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:55.313+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:55.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:55 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:55.313+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:19:55.852 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:19:55 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:19:55.414+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:19:55.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:55 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:55.355+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:55.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:55 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:55.355+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:19:56.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:55 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:55.538+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:56.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:55 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:55.869+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:56.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:55 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:55.869+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:19:56.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:55 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:55.980+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:56.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:56 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:56.164+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:56.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:56 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:56.164+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:19:56.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:56 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:56.312+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:56.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:56 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:56.312+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:19:56.852 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:56 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4[43925]: 2024-04-25T00:19:56.455+0000 7fe4b890a700 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:26.983951+0000 front 2024-04-25T00:19:26.983979+0000 (oldest deadline 2024-04-25T00:19:56.384537+0000) 2024-04-25T00:19:56.853 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:19:56 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:19:56.407+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:19:56.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:56 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:56.374+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:56.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:56 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:56.374+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:19:57.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:56 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:56.579+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:57.286 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:56 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:56.965+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:57.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:56 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:56.830+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:57.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:56 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:56.830+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:19:57.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:57 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:57.159+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:57.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:57 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:57.159+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:19:57.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:57 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:57.265+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:57.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:57 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:57.265+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:19:57.750 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:19:57 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:19:57.409+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:19:57.750 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:57 smithi173 podman[68863]: 2024-04-25 00:19:57.499626405 +0000 UTC m=+5.507703094 container died b505068aca92e704cbf7a403167cc5586399dfbf162caa9556f3727113965fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T00:19:57.751 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:57 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:57.373+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:57.751 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:57 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:57.373+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:19:58.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:57 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:57.590+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:58.102 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:57 smithi173 podman[68863]: 2024-04-25 00:19:57.840598781 +0000 UTC m=+5.848675468 container cleanup b505068aca92e704cbf7a403167cc5586399dfbf162caa9556f3727113965fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T00:19:58.102 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:57 smithi173 bash[68863]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4 2024-04-25T00:19:58.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:57 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:57.870+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:58.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:57 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:57.870+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:19:58.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:57 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:57.993+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:58.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:57 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:57.993+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:19:58.386 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:58 smithi173 podman[68880]: 2024-04-25 00:19:58.108838952 +0000 UTC m=+0.605393461 container remove b505068aca92e704cbf7a403167cc5586399dfbf162caa9556f3727113965fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T00:19:58.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:58 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:58.180+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:58.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:58 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:58.180+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:19:58.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:58 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:58.305+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:58.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:58 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:58.305+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:19:58.645 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:19:58 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:19:58.385+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:19:58.646 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:58 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:58.417+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:58.646 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:58 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:58.417+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:19:58.646 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:58 smithi173 podman[68968]: 2024-04-25 00:19:58.415655845 +0000 UTC m=+0.018063417 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:58.646 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:58 smithi173 podman[68968]: 2024-04-25 00:19:58.534978739 +0000 UTC m=+0.137386301 container create 8ee562aac00ddef67fa738c227405a28105b2695360744d67627ade031a94cbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T00:19:59.017 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:58 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:58.598+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:19:59.065 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:58 smithi173 podman[68968]: 2024-04-25 00:19:58.771780583 +0000 UTC m=+0.374188145 container init 8ee562aac00ddef67fa738c227405a28105b2695360744d67627ade031a94cbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T00:19:59.065 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:58 smithi173 podman[68968]: 2024-04-25 00:19:58.774986305 +0000 UTC m=+0.377393877 container start 8ee562aac00ddef67fa738c227405a28105b2695360744d67627ade031a94cbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:19:59.065 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:58 smithi173 podman[68968]: 2024-04-25 00:19:58.87200386 +0000 UTC m=+0.474411450 container attach 8ee562aac00ddef67fa738c227405a28105b2695360744d67627ade031a94cbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T00:19:59.065 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 25 00:19:59 smithi173 podman[68968]: 2024-04-25 00:19:59.065206157 +0000 UTC m=+0.667613711 container died 8ee562aac00ddef67fa738c227405a28105b2695360744d67627ade031a94cbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T00:19:59.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:58 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:58.844+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:19:59.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:58 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:58.844+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:19:59.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:59 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:59.033+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:19:59.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:19:59 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:19:59.033+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:19:59.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:59 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:59.189+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:19:59.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:59 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:19:59.189+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:19:59.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:59 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:59.333+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:19:59.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:19:59 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:19:59.333+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:19:59.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:19:59 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:19:59.347+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:19:59.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:59 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:59.370+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:19:59.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:19:59 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:19:59.370+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:19:59.731 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T00:19:59.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:19:59.770 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T00:19:59.770 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T00:19:59.770 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.5 2024-04-25T00:19:59.814 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:19:59 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:19:59.550+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:00.077 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:19:59 smithi188 systemd[1]: Stopping Ceph osd.5 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:20:00.077 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:00 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:00.040+0000 7f9d76764700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:00.077 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:00 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:00.040+0000 7f9d76764700 -1 osd.5 90 *** Got signal Terminated *** 2024-04-25T00:20:00.077 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:00 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:00.040+0000 7f9d76764700 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:20:00.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:59 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:59.878+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:00.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:19:59 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:19:59.878+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:00.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:00.066+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:00.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:00.066+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:00.380 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:00 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:00.155+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:20:00.380 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:00 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:00.155+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:20:00.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:00 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:00.591+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:00.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:00 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:00.380+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:00.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:00 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:00.380+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:00.852 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:00 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:00.374+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:00.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:00 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:00.406+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:00.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:00 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:00.406+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:01.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:00.882+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:01.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:00.882+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:01.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:01 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:01.057+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:01.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:01 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:01.057+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:01.411 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:01 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:01.120+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:20:01.411 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:01 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:01.120+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:20:01.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:01 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:01.563+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:01.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:01 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:01.410+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:01.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:01 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:01.410+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:01.852 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:01 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:01.354+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:01.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:01 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:01.359+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:01.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:01 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:01.359+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:02.286 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:02.013+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:02.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:02.013+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:02.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:01 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:01.873+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:02.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:01 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:01.873+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:02.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:01 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:01.873+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:02.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:02 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:02.160+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:20:02.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:02 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:02.160+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:20:02.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:02 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:02.393+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:02.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:02 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:02.393+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:02.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:02 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:02.370+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:02.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:02 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:02.370+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:02.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:02 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:02.344+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:02.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:02 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:02.344+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:03.017 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:02 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:02.608+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:03.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:02.884+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:03.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:02.884+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:03.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:02.884+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:03.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:02.981+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:03.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:02.981+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:03.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:02 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:02.981+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:03.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:03 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:03.192+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:20:03.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:03 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:03.192+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:20:03.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:03 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:03.440+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:03.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:03 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:03.440+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:03.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:03 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:03.414+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:03.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:03 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:03.414+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:03.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:03 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:03.337+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:03.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:03 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:03.337+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:04.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:03 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:03.649+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:04.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:03 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:03.649+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:04.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:03.891+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:04.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:03.891+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:04.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:03.891+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:04.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:03.941+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:04.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:03.941+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:04.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:03 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:03.941+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:04.479 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:04 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:04.149+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.066333+0000 front 2024-04-25T00:19:22.066203+0000 (oldest deadline 2024-04-25T00:19:46.766009+0000) 2024-04-25T00:20:04.479 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:04 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5[45229]: 2024-04-25T00:20:04.149+0000 7f9d72d58700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.868330+0000 front 2024-04-25T00:19:31.868384+0000 (oldest deadline 2024-04-25T00:19:54.668841+0000) 2024-04-25T00:20:04.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:04 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:04.434+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:04.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:04 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:04.434+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:04.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:04 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:04.343+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:04.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:04 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:04.343+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:04.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:04 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:04.674+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:04.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:04 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:04.674+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:04.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:04 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:04.479+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:04.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:04 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:04.479+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:05.238 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:04.907+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:05.239 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:04.907+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:05.239 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:04.907+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:05.239 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:04.965+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:05.239 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:04.965+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:05.239 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:04.965+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:05.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:05 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:05.474+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:05.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:05 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:05.474+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:05.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:05 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:05.314+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:05.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:05 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:05.314+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:05.708 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:05 smithi188 podman[70087]: 2024-04-25 00:20:05.423399738 +0000 UTC m=+5.463393823 container died c0255818ac96392ad586292aaaaabf04f8240f7415900190521e9c4d0f5b36a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:20:05.709 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:05 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:05.660+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:05.709 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:05 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:05.660+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:05.709 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:05 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:05.474+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:05.709 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:05 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:05.474+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:06.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:05 smithi188 podman[70087]: 2024-04-25 00:20:05.815799064 +0000 UTC m=+5.855793137 container cleanup c0255818ac96392ad586292aaaaabf04f8240f7415900190521e9c4d0f5b36a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T00:20:06.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:05 smithi188 bash[70087]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5 2024-04-25T00:20:06.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:05.914+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:06.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:05.914+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:06.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:05.914+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:06.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:05.991+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:06.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:05.991+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:06.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:05 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:05.991+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:06.394 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:06 smithi188 podman[70105]: 2024-04-25 00:20:06.092355644 +0000 UTC m=+0.670134359 container remove c0255818ac96392ad586292aaaaabf04f8240f7415900190521e9c4d0f5b36a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T00:20:06.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:06.282+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:06.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:06.282+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:06.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:06.474+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:06.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:06.475+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:06.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:06 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:06.475+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:06.645 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:06 smithi188 podman[70198]: 2024-04-25 00:20:06.298248162 +0000 UTC m=+0.018030958 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:20:06.645 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:06 smithi188 podman[70198]: 2024-04-25 00:20:06.426698068 +0000 UTC m=+0.146480881 container create 33eb21b16d36d18d1303da0931d0c2db87ef18758d07bdc8f6a214025858efe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:20:06.646 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:06.632+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:06.646 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:06.632+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:06.646 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:06.463+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:06.646 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:06.463+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:06.646 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:06 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:06.463+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:06.934 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:06 smithi188 podman[70198]: 2024-04-25 00:20:06.645694057 +0000 UTC m=+0.365476835 container init 33eb21b16d36d18d1303da0931d0c2db87ef18758d07bdc8f6a214025858efe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:20:06.934 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:06 smithi188 podman[70198]: 2024-04-25 00:20:06.648937336 +0000 UTC m=+0.368720107 container start 33eb21b16d36d18d1303da0931d0c2db87ef18758d07bdc8f6a214025858efe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-25T00:20:06.934 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:06 smithi188 podman[70198]: 2024-04-25 00:20:06.77099541 +0000 UTC m=+0.490778193 container attach 33eb21b16d36d18d1303da0931d0c2db87ef18758d07bdc8f6a214025858efe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T00:20:07.268 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 25 00:20:06 smithi188 podman[70198]: 2024-04-25 00:20:06.935030435 +0000 UTC m=+0.654813220 container died 33eb21b16d36d18d1303da0931d0c2db87ef18758d07bdc8f6a214025858efe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T00:20:07.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:06.864+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:07.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:06.864+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:07.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:06.864+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:07.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:06.983+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:07.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:06.983+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:07.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:06 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:06.983+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:07.583 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:07 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:07.462+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:07.583 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:07 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:07.462+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:07.583 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:07 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:07.462+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:07.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T00:20:07.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:07 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:07.298+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:07.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:07 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:07.298+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:07.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:07 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:07.439+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:07.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:07 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:07.439+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:07.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:07 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:07.439+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:07.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:20:07.681 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T00:20:07.681 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T00:20:07.681 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.6 2024-04-25T00:20:07.976 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:07 smithi110 systemd[1]: Stopping Ceph osd.6 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:20:07.976 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:07.850+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:07.976 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:07.850+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:07.976 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:07.850+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:07.976 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:07.975+0000 7fe5df60d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:07.976 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:07.975+0000 7fe5df60d700 -1 osd.6 90 *** Got signal Terminated *** 2024-04-25T00:20:08.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:07 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:07.674+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:08.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:07 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:07.674+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:08.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:07 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:07.975+0000 7fe5df60d700 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:20:08.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:08 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:08.005+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:08.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:08 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:08.005+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:08.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:08 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:08.005+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:08.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:08 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:08.249+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:08.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:08 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:08.249+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:08.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:08 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:08.466+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:08.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:08 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:08.466+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:08.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:08 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:08.466+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:08.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:08 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:08.695+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:08.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:08 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:08.695+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:08.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:08 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:08.464+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:08.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:08 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:08.464+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:08.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:08 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:08.464+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:09.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:08 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:08.832+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:09.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:08 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:08.832+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:09.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:08 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:08.832+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:09.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:09 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:09.040+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:09.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:09 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:09.040+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:09.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:09 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:09.040+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:09.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:09 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:09.248+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:09.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:09 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:09.248+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:09.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:09 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:09.467+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:09.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:09 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:09.467+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:09.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:09 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:09.467+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:09.745 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:09 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:09.462+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:09.745 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:09 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:09.462+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:09.745 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:09 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:09.462+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:10.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:09 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:09.744+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:10.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:09 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:09.744+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:10.289 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:09 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:09.833+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:10.290 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:09 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:09.833+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:10.290 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:09 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:09.833+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:10.290 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:10.076+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:10.290 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:10.076+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:10.290 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:10.076+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:10.290 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:10.076+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:10.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:10.264+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:10.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:10.264+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:10.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:10.503+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:10.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:10.503+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:10.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:10 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:10.503+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:10.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:10.509+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:10.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:10.509+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:10.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:10.509+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:11.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:10.794+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:11.269 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:10 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:10.794+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:11.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:10.870+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:11.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:10.870+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:11.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:10 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:10.870+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:11.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:11 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:11.095+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:11.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:11 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:11.095+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:11.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:11 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:11.095+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:11.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:11 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:11.095+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:11.509 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:11 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:11.243+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:11.509 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:11 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:11.243+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:11.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:11.476+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:11.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:11.476+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:11.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:11.476+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:11.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:11.476+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:11.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:11 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:11.508+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:11.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:11 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:11.508+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:11.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:11 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:11.508+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:12.135 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:11 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:11.842+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:12.135 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:11 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:11.842+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:12.135 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:11 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:11.842+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:12.135 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:11 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:11.842+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.726713+0000 front 2024-04-25T00:19:45.726782+0000 (oldest deadline 2024-04-25T00:20:11.026897+0000) 2024-04-25T00:20:12.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:11.835+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:12.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:11.835+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:12.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:11 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:11.835+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:12.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:12 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:12.134+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:12.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:12 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:12.134+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:12.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:12 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:12.134+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:12.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:12 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:12.134+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:12.549 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:12.206+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:12.550 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:12.206+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:12.550 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:12.206+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:12.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:12.432+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:12.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:12.432+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:12.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:12.432+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:12.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:12.432+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:12.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:12.549+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:12.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:12.549+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:12.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:12 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:12.549+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:13.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:12.865+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:13.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:12.865+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:13.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:12 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:12.865+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:13.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:12 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:12.874+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:19.621962+0000 front 2024-04-25T00:19:19.622007+0000 (oldest deadline 2024-04-25T00:19:44.822735+0000) 2024-04-25T00:20:13.286 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:12 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:12.874+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:27.624370+0000 front 2024-04-25T00:19:27.624241+0000 (oldest deadline 2024-04-25T00:19:53.424879+0000) 2024-04-25T00:20:13.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:12 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:12.874+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:38.726133+0000 front 2024-04-25T00:19:38.726119+0000 (oldest deadline 2024-04-25T00:20:01.625871+0000) 2024-04-25T00:20:13.287 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:12 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6[57206]: 2024-04-25T00:20:12.874+0000 7fe5dbc01700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.726713+0000 front 2024-04-25T00:19:45.726782+0000 (oldest deadline 2024-04-25T00:20:11.026897+0000) 2024-04-25T00:20:13.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:13 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:13.114+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:13.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:13 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:13.114+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:13.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:13 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:13.114+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:13.287 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:13 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:13.114+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:13.572 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:13.234+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:13.572 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:13.234+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:13.572 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:13.234+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:13.709 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:13 smithi110 podman[94194]: 2024-04-25 00:20:13.409619468 +0000 UTC m=+5.538875667 container died 3f772af33bfb57a08659537ef75ac7ae5b2a7f7a932551f105e5e85cbd850e0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:20:13.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:13.416+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:13.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:13.416+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:13.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:13.416+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:13.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:13.416+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:13.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:13.571+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:13.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:13.571+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:13.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:13.571+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:13.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:13.571+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:14.036 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:13 smithi110 podman[94194]: 2024-04-25 00:20:13.800413529 +0000 UTC m=+5.929669728 container cleanup 3f772af33bfb57a08659537ef75ac7ae5b2a7f7a932551f105e5e85cbd850e0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:20:14.037 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:13 smithi110 bash[94194]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6 2024-04-25T00:20:14.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:13.867+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:14.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:13.867+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:14.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:13 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:13.867+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:14.346 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:14 smithi110 podman[94211]: 2024-04-25 00:20:14.04332468 +0000 UTC m=+0.633235753 container remove 3f772af33bfb57a08659537ef75ac7ae5b2a7f7a932551f105e5e85cbd850e0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T00:20:14.347 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:14 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:14.146+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:14.347 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:14 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:14.146+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:14.347 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:14 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:14.146+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:14.347 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:14 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:14.146+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:14.524 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:14.204+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:14.524 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:14.204+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:14.525 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:14.204+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:14.540 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:14.383+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:14.540 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:14.383+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:14.541 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:14.383+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:14.541 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:14.383+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:14.622 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:14 smithi110 podman[94299]: 2024-04-25 00:20:14.249914367 +0000 UTC m=+0.017774188 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:20:14.622 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:14 smithi110 podman[94299]: 2024-04-25 00:20:14.369294963 +0000 UTC m=+0.137154771 container create d57da9ff8a167ae0386be6ec91c8e8f5054e1cead7ae389f6b8538053be7cbb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:20:14.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:14.524+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:14.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:14.524+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:14.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:14.524+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:14.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:14 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:14.524+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:14.910 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:14 smithi110 podman[94299]: 2024-04-25 00:20:14.621995654 +0000 UTC m=+0.389855462 container init d57da9ff8a167ae0386be6ec91c8e8f5054e1cead7ae389f6b8538053be7cbb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9) 2024-04-25T00:20:14.910 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:14 smithi110 podman[94299]: 2024-04-25 00:20:14.625050116 +0000 UTC m=+0.392909927 container start d57da9ff8a167ae0386be6ec91c8e8f5054e1cead7ae389f6b8538053be7cbb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:20:14.910 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:14 smithi110 podman[94299]: 2024-04-25 00:20:14.738666784 +0000 UTC m=+0.506526600 container attach d57da9ff8a167ae0386be6ec91c8e8f5054e1cead7ae389f6b8538053be7cbb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T00:20:15.258 INFO:journalctl@ceph.osd.6.smithi110.stdout:Apr 25 00:20:14 smithi110 podman[94299]: 2024-04-25 00:20:14.910692174 +0000 UTC m=+0.678551989 container died d57da9ff8a167ae0386be6ec91c8e8f5054e1cead7ae389f6b8538053be7cbb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T00:20:15.258 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:15 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:15.131+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:15.258 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:15 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:15.131+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:15.258 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:15 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:15.131+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:15.258 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:15 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:15.131+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:15.267 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:14.831+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:15.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:14.831+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:15.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:14 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:14.831+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:15.527 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:15.176+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:15.528 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:15.176+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:15.528 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:15.176+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:15.597 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T00:20:15.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:20:15.635 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T00:20:15.636 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T00:20:15.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.7 2024-04-25T00:20:15.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:15 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:15.433+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:15.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:15 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:15.433+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:15.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:15 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:15.433+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:15.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:15 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:15.433+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:15.852 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:15 smithi173 systemd[1]: Stopping Ceph osd.7 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:20:15.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:15.527+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:15.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:15.527+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:15.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:15.527+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:15.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:15.527+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:16.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:15 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:15.804+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:16.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:15 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:15.804+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:16.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:15 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:15.804+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:16.352 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:15.947+0000 7fe07fc73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:16.352 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:15.947+0000 7fe07fc73700 -1 osd.7 90 *** Got signal Terminated *** 2024-04-25T00:20:16.353 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:15.947+0000 7fe07fc73700 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:20:16.353 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:16.129+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:16.353 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:16.129+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:16.353 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:16.129+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:16.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:16 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:16.178+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:16.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:16 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:16.178+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:16.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:16 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:16.178+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:16.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:16 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:16.178+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:16.612 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:16.555+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:16.612 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:16.555+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:16.613 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:16.555+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:16.613 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:16 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:16.555+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:16.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:16 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:16.428+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:16.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:16 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:16.428+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:16.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:16 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:16.428+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:16.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:16 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:16.428+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:17.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:16 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:16.807+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:17.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:16 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:16.807+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:17.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:16 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:16.807+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:17.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:17 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:17.136+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:17.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:17 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:17.136+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:17.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:17 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:17.136+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:17.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:17 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:17.136+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:17.537 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:17 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:17.144+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:17.537 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:17 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:17.144+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:17.537 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:17 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:17.144+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:17.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:17 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:17.434+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:17.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:17 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:17.434+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:17.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:17 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:17.434+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:17.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:17 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:17.434+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:17.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:17 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:17.588+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:17.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:17 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:17.588+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:17.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:17 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:17.588+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:17.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:17 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:17.588+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:18.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:17 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:17.779+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:18.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:17 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:17.779+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:18.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:17 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:17.779+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:18.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:18 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:18.155+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:18.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:18 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:18.155+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:18.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:18 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:18.155+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:18.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:18 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:18.155+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:18.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:18 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:18.126+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:18.602 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:18 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:18.126+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:18.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:18 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:18.126+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:18.766 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:18 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:18.447+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:18.766 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:18 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:18.447+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:18.766 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:18 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:18.447+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:18.766 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:18 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:18.447+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:19.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:18 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:18.766+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:19.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:18 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:18.766+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:19.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:18 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:18.766+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:19.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:18 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:18.615+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:19.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:18 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:18.615+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:19.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:18 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:18.615+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:19.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:18 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:18.615+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:19.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:18 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:18.615+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:19.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:19 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:19.130+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:19.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:19 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:19.130+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:19.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:19 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:19.130+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:19.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:19 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:19.130+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:19.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:19 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:19.130+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:19.587 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:19.149+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:19.588 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:19.149+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:19.588 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:19.149+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:19.748 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:19 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:19.406+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:19.748 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:19 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:19.406+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:19.748 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:19 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:19.406+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:19.748 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:19 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:19.406+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:19.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:19.587+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:19.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:19.587+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:19.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:19.587+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:19.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:19.587+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:19.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:19.587+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:20.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:19 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:19.747+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:20.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:19 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:19.747+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:20.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:19 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:19.747+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:20.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:20 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:20.149+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:20.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:20 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:20.149+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:20.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:20 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:20.149+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:20.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:20 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:20.149+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:20.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:20 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:20.149+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:20.548 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:20 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:20.144+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.019526+0000 front 2024-04-25T00:19:31.019559+0000 (oldest deadline 2024-04-25T00:19:53.919717+0000) 2024-04-25T00:20:20.548 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:20 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:20.144+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.321147+0000 front 2024-04-25T00:19:40.321300+0000 (oldest deadline 2024-04-25T00:20:02.021225+0000) 2024-04-25T00:20:20.548 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:20 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7[50616]: 2024-04-25T00:20:20.144+0000 7fe07c267700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.222624+0000 front 2024-04-25T00:19:47.222514+0000 (oldest deadline 2024-04-25T00:20:11.922635+0000) 2024-04-25T00:20:20.714 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:20.446+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:20.714 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:20.446+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:20.714 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:20.446+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:20.714 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:20.446+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:20.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:20 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:20.548+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:20.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:20 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:20.548+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:20.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:20 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:20.548+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:20.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:20 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:20.548+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:20.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:20 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:20.548+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:21.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:20.713+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:21.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:20.713+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:21.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:20 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:20.713+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:21.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:21.156+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:21.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:21.156+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:21.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:21.156+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:21.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:21.156+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:21.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:21 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:21.156+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:21.664 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:21 smithi173 podman[69074]: 2024-04-25 00:20:21.373743187 +0000 UTC m=+5.545080086 container died f3c595acd8c8503dfb2c5f082b8bf1283c617f118eb4e4e5026fee8cfdbc7728 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:20:21.665 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:21 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:21.518+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:21.665 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:21 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:21.518+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:21.665 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:21 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:21.518+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:21.665 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:21 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:21.518+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:21.665 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:21 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:21.518+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:21.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:21 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:21.691+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:21.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:21 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:21.691+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:21.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:21 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:21.691+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:21.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:21 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:21.691+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.324400+0000 front 2024-04-25T00:19:51.324326+0000 (oldest deadline 2024-04-25T00:20:20.724625+0000) 2024-04-25T00:20:21.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:21 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:21.491+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:21.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:21 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:21.491+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:21.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:21 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:21.491+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:21.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:21 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:21.491+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:22.032 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:21 smithi173 podman[69074]: 2024-04-25 00:20:21.76406761 +0000 UTC m=+5.935404509 container cleanup f3c595acd8c8503dfb2c5f082b8bf1283c617f118eb4e4e5026fee8cfdbc7728 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:20:22.032 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:21 smithi173 bash[69074]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7 2024-04-25T00:20:22.349 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:22 smithi173 podman[69104]: 2024-04-25 00:20:22.032096634 +0000 UTC m=+0.655704336 container remove f3c595acd8c8503dfb2c5f082b8bf1283c617f118eb4e4e5026fee8cfdbc7728 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:20:22.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:22 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:22.113+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:22.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:22 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:22.113+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:22.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:22 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:22.113+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:22.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:22 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:22.113+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:22.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:22 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:22.113+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:22 smithi173 podman[69190]: 2024-04-25 00:20:22.349534672 +0000 UTC m=+0.112849042 container create 577cc1027d4d7cb434dcaee84fdad536339d465cf2d18183ba55b440c09cf0b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:22 smithi173 podman[69190]: 2024-04-25 00:20:22.25463099 +0000 UTC m=+0.017945368 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:22 smithi173 podman[69190]: 2024-04-25 00:20:22.5687675 +0000 UTC m=+0.332081871 container init 577cc1027d4d7cb434dcaee84fdad536339d465cf2d18183ba55b440c09cf0b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:22 smithi173 podman[69190]: 2024-04-25 00:20:22.571856563 +0000 UTC m=+0.335170942 container start 577cc1027d4d7cb434dcaee84fdad536339d465cf2d18183ba55b440c09cf0b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:22 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:22.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:22 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:22.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:22 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:22.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:22 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:22.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:22.603 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:22 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:22.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:22.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:22.672+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:22.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:22.672+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:22.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:22.672+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:22.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:22.672+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.324400+0000 front 2024-04-25T00:19:51.324326+0000 (oldest deadline 2024-04-25T00:20:20.724625+0000) 2024-04-25T00:20:22.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:22.478+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:22.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:22.478+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:22.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:22.478+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:22.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:22.478+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:22.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:22 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:22.478+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:22.868 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:22 smithi173 podman[69190]: 2024-04-25 00:20:22.746807147 +0000 UTC m=+0.510121521 container attach 577cc1027d4d7cb434dcaee84fdad536339d465cf2d18183ba55b440c09cf0b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T00:20:23.173 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 25 00:20:22 smithi173 podman[69190]: 2024-04-25 00:20:22.868847166 +0000 UTC m=+0.632161538 container died 577cc1027d4d7cb434dcaee84fdad536339d465cf2d18183ba55b440c09cf0b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T00:20:23.458 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T00:20:23.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:20:23.497 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T00:20:23.497 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T00:20:23.498 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.8 2024-04-25T00:20:23.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:23.143+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:23.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:23.143+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:23.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:23.143+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:23.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:23.143+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:23.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:23 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:23.143+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:23.827 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:23 smithi188 systemd[1]: Stopping Ceph osd.8 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:20:23.827 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:23.671+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:23.827 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:23.671+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:23.827 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:23.671+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:23.827 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:23.671+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.324400+0000 front 2024-04-25T00:19:51.324326+0000 (oldest deadline 2024-04-25T00:20:20.724625+0000) 2024-04-25T00:20:23.827 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:23.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:23.827 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:23.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:23.827 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:23.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:23.828 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:23.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:23.828 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:23.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:23.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:23 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:23.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:23.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:23 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:23.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:23.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:23 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:23.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:23.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:23 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:23.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:23.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:23 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:23.541+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:24.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:23.826+0000 7f5a860a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:24.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:23.826+0000 7f5a860a4700 -1 osd.8 90 *** Got signal Terminated *** 2024-04-25T00:20:24.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:23 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:23.826+0000 7f5a860a4700 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:20:24.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:24 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:24.129+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:24.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:24 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:24.129+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:24.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:24 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:24.129+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:24.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:24 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:24.129+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:24.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:24 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:24.129+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:24.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:24 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:24.582+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:24.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:24 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:24.582+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:24.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:24 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:24.582+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:24.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:24 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:24.582+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:24.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:24 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:24.582+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:24.980 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:24.661+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:24.980 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:24.661+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:24.980 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:24.661+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:24.980 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:24.661+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.324400+0000 front 2024-04-25T00:19:51.324326+0000 (oldest deadline 2024-04-25T00:20:20.724625+0000) 2024-04-25T00:20:24.980 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:24.568+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:24.980 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:24.568+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:24.981 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:24.568+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:24.981 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:24.568+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:24.981 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:24 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:24.568+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:25.438 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:25 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:25.151+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:25.438 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:25 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:25.151+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:25.438 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:25 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:25.151+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:25.438 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:25 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:25.151+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:25.438 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:25 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:25.151+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:25.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:25 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:25.568+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:25.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:25 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:25.568+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:25.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:25 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:25.568+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:25.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:25 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:25.568+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:25.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:25 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:25.568+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:25.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:25 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:25.568+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:26.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:25.586+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:26.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:25.586+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:26.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:25.586+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:26.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:25.586+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:26.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:25.586+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:26.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:25.687+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:26.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:25.687+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:26.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:25.687+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:26.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:25 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:25.687+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.324400+0000 front 2024-04-25T00:19:51.324326+0000 (oldest deadline 2024-04-25T00:20:20.724625+0000) 2024-04-25T00:20:26.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:26.187+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:26.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:26.187+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:26.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:26.187+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:26.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:26.187+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:26.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:26 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:26.187+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:26.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:26 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:26.594+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:26.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:26 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:26.594+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:26.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:26 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:26.594+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:26.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:26 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:26.594+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:26.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:26 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:26.594+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:26.853 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:26 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:26.594+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:27.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:26.734+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:27.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:26.734+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:27.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:26.734+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:27.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:26.734+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.324400+0000 front 2024-04-25T00:19:51.324326+0000 (oldest deadline 2024-04-25T00:20:20.724625+0000) 2024-04-25T00:20:27.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:26.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:27.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:26.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:27.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:26.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:27.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:26.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:27.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:26.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:27.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:26.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:27.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:27 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:27.148+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:27.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:27 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:27.148+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:27.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:27 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:27.148+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:27.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:27 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:27.148+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:27.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:27 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:27.148+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:28.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:27.713+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:28.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:27.713+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:28.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:27.713+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:28.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:27.713+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.324400+0000 front 2024-04-25T00:19:51.324326+0000 (oldest deadline 2024-04-25T00:20:20.724625+0000) 2024-04-25T00:20:28.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:27.591+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:28.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:27.591+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:28.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:27.591+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:28.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:27.591+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:28.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:27.591+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:28.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:27 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:27.591+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:28.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:27.641+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:28.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:27.641+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:28.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:27.641+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:28.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:27.641+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:28.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:27.641+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:28.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:27 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:27.641+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:28.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:28 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:28.179+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:28.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:28 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:28.179+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:28.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:28 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:28.179+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:28.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:28 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:28.179+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:28.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:28 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:28.179+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:29.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:28.679+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.820456+0000 front 2024-04-25T00:19:20.820367+0000 (oldest deadline 2024-04-25T00:19:46.720341+0000) 2024-04-25T00:20:29.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:28.679+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:37.222508+0000 front 2024-04-25T00:19:37.222554+0000 (oldest deadline 2024-04-25T00:20:03.122608+0000) 2024-04-25T00:20:29.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:28.679+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:45.423651+0000 front 2024-04-25T00:19:45.423673+0000 (oldest deadline 2024-04-25T00:20:11.323755+0000) 2024-04-25T00:20:29.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8[51775]: 2024-04-25T00:20:28.679+0000 7f5a7e690700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.324400+0000 front 2024-04-25T00:19:51.324326+0000 (oldest deadline 2024-04-25T00:20:20.724625+0000) 2024-04-25T00:20:29.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:28.603+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:29.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:28.603+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:29.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:28.603+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:29.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:28.603+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:29.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:28.603+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:29.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:28 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:28.603+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:29.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:28 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:28.669+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:29.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:28 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:28.669+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:29.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:28 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:28.669+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:29.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:28 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:28.669+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:29.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:28 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:28.669+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:29.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:28 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:28.669+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:29.460 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:29 smithi188 podman[70315]: 2024-04-25 00:20:29.187012708 +0000 UTC m=+5.496135550 container died d03a6a594ba584b10ea875d8bfbb6925e38cbce4bcc41796d86d222f498be3a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T00:20:29.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:29 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:29.221+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:29.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:29 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:29.221+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:29.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:29 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:29.221+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:29.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:29 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:29.221+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:29.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:29 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:29.221+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:29.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:29 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:29.221+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.399190+0000 front 2024-04-25T00:20:04.399106+0000 (oldest deadline 2024-04-25T00:20:28.498897+0000) 2024-04-25T00:20:29.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:29 smithi188 podman[70315]: 2024-04-25 00:20:29.559704012 +0000 UTC m=+5.868826850 container cleanup d03a6a594ba584b10ea875d8bfbb6925e38cbce4bcc41796d86d222f498be3a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T00:20:29.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:29 smithi188 bash[70315]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8 2024-04-25T00:20:29.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:29 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:29.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:29.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:29 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:29.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:29.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:29 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:29.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:29.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:29 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:29.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:29.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:29 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:29.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:29.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:29 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:29.578+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:30.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:29 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:29.676+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:30.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:29 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:29.676+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:30.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:29 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:29.676+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:30.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:29 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:29.676+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:30.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:29 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:29.676+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:30.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:29 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:29.676+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:30.179 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:29 smithi188 podman[70336]: 2024-04-25 00:20:29.86130591 +0000 UTC m=+0.668116999 container remove d03a6a594ba584b10ea875d8bfbb6925e38cbce4bcc41796d86d222f498be3a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:20:30.431 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:30 smithi188 podman[70424]: 2024-04-25 00:20:30.082779275 +0000 UTC m=+0.017664757 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:20:30.431 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:30 smithi188 podman[70424]: 2024-04-25 00:20:30.195494802 +0000 UTC m=+0.130380274 container create 9601f8196c900018ccd53848ec00b825634667403266824e830fd9dfffcc2420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-25T00:20:30.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:30 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:30.254+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:30.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:30 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:30.254+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:30.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:30 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:30.254+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:30.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:30 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:30.254+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:30.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:30 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:30.254+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:30.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:30 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:30.254+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.399190+0000 front 2024-04-25T00:20:04.399106+0000 (oldest deadline 2024-04-25T00:20:28.498897+0000) 2024-04-25T00:20:30.730 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:30 smithi188 podman[70424]: 2024-04-25 00:20:30.431787793 +0000 UTC m=+0.366673265 container init 9601f8196c900018ccd53848ec00b825634667403266824e830fd9dfffcc2420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:20:30.730 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:30 smithi188 podman[70424]: 2024-04-25 00:20:30.435730241 +0000 UTC m=+0.370615724 container start 9601f8196c900018ccd53848ec00b825634667403266824e830fd9dfffcc2420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:20:30.730 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:30 smithi188 podman[70424]: 2024-04-25 00:20:30.531673988 +0000 UTC m=+0.466559460 container attach 9601f8196c900018ccd53848ec00b825634667403266824e830fd9dfffcc2420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T00:20:30.730 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:30 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:30.614+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:30.730 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:30 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:30.614+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:30.730 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:30 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:30.614+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:30.731 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:30 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:30.614+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:30.731 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:30 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:30.614+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:30.731 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:30 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:30.614+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:31.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 25 00:20:30 smithi188 podman[70424]: 2024-04-25 00:20:30.730619595 +0000 UTC m=+0.665505068 container died 9601f8196c900018ccd53848ec00b825634667403266824e830fd9dfffcc2420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T00:20:31.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:30 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:30.720+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:31.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:30 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:30.720+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:31.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:30 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:30.720+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:31.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:30 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:30.720+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:31.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:30 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:30.720+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:31.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:30 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:30.720+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:31.373 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T00:20:31.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:20:31.414 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T00:20:31.414 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T00:20:31.414 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.9 2024-04-25T00:20:31.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.265+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:31.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.265+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:31.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.265+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:31.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.265+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:31.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.265+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:31.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.265+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.399190+0000 front 2024-04-25T00:20:04.399106+0000 (oldest deadline 2024-04-25T00:20:28.498897+0000) 2024-04-25T00:20:31.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 systemd[1]: Stopping Ceph osd.9 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:20:32.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:31 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:31.595+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:32.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:31 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:31.595+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:32.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:31 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:31.595+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:32.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:31 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:31.595+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:32.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:31 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:31.595+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:32.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:31 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:31.595+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:32.036 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.743+0000 7f6f1100c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:32.036 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.743+0000 7f6f1100c700 -1 osd.9 90 *** Got signal Terminated *** 2024-04-25T00:20:32.036 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:31 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:31.743+0000 7f6f1100c700 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:20:32.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:31 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:31.700+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:32.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:31 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:31.700+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:32.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:31 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:31.700+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:32.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:31 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:31.700+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:32.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:31 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:31.700+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:32.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:31 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:31.700+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:32.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:32.260+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:32.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:32.260+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:32.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:32.260+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:32.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:32.260+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:32.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:32.260+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:32.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:32 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:32.260+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.399190+0000 front 2024-04-25T00:20:04.399106+0000 (oldest deadline 2024-04-25T00:20:28.498897+0000) 2024-04-25T00:20:33.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:32 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:32.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:33.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:32 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:32.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:33.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:32 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:32.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:33.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:32 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:32.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:33.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:32 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:32.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:33.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:32 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:32.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:33.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:32 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:32.731+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:33.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:32 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:32.731+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:33.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:32 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:32.731+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:33.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:32 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:32.731+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:33.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:32 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:32.731+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:33.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:32 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:32.731+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:33.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:33 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:33.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:33.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:33 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:33.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:33.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:33 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:33.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:33.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:33 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:33.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:33.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:33 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:33.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:33.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:33 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:33.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.399190+0000 front 2024-04-25T00:20:04.399106+0000 (oldest deadline 2024-04-25T00:20:28.498897+0000) 2024-04-25T00:20:34.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:33 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:33.550+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:34.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:33 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:33.550+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:34.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:33 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:33.550+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:34.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:33 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:33.550+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:34.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:33 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:33.550+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:34.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:33 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:33.550+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:34.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:33 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:33.769+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:34.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:33 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:33.769+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:34.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:33 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:33.769+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:34.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:33 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:33.769+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:34.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:33 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:33.769+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:34.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:33 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:33.769+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:34.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:34 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:34.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:34.536 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:34 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:34.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:34.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:34 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:34.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:34.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:34 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:34.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:34.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:34 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:34.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:34.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:34 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:34.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.399190+0000 front 2024-04-25T00:20:04.399106+0000 (oldest deadline 2024-04-25T00:20:28.498897+0000) 2024-04-25T00:20:34.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:34 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:34.251+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:08.499352+0000 front 2024-04-25T00:20:08.499375+0000 (oldest deadline 2024-04-25T00:20:33.799494+0000) 2024-04-25T00:20:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:34 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:34.559+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:34 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:34.559+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:34 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:34.559+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:34 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:34.559+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:34 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:34.559+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:35.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:34 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:34.559+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:35.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:34 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:34.786+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:35.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:34 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:34.786+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:35.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:34 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:34.786+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:35.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:34 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:34.786+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:35.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:34 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:34.786+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:35.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:34 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:34.786+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:35.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:35 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:35.219+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:35.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:35 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:35.219+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:35.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:35 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:35.219+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:35.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:35 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:35.219+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:35.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:35 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:35.219+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:35.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:35 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:35.219+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.399190+0000 front 2024-04-25T00:20:04.399106+0000 (oldest deadline 2024-04-25T00:20:28.498897+0000) 2024-04-25T00:20:35.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:35 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:35.219+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:08.499352+0000 front 2024-04-25T00:20:08.499375+0000 (oldest deadline 2024-04-25T00:20:33.799494+0000) 2024-04-25T00:20:36.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:35.581+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:36.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:35.581+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:36.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:35.581+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:36.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:35.581+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:36.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:35.581+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:36.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:35.581+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:36.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:35 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:35.745+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:36.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:35 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:35.745+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:36.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:35 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:35.745+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:36.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:35 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:35.745+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:36.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:35 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:35.745+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:36.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:35 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:35.745+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:36.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:36 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:36.170+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:22.491980+0000 front 2024-04-25T00:19:22.492043+0000 (oldest deadline 2024-04-25T00:19:47.192214+0000) 2024-04-25T00:20:36.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:36 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:36.170+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.094501+0000 front 2024-04-25T00:19:31.094399+0000 (oldest deadline 2024-04-25T00:19:56.994357+0000) 2024-04-25T00:20:36.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:36 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:36.170+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:36.994830+0000 front 2024-04-25T00:19:36.995011+0000 (oldest deadline 2024-04-25T00:20:02.894782+0000) 2024-04-25T00:20:36.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:36 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:36.170+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:46.896306+0000 front 2024-04-25T00:19:46.896427+0000 (oldest deadline 2024-04-25T00:20:09.796415+0000) 2024-04-25T00:20:36.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:36 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:36.170+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:54.398124+0000 front 2024-04-25T00:19:54.398206+0000 (oldest deadline 2024-04-25T00:20:18.498050+0000) 2024-04-25T00:20:36.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:36 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:36.170+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.399190+0000 front 2024-04-25T00:20:04.399106+0000 (oldest deadline 2024-04-25T00:20:28.498897+0000) 2024-04-25T00:20:36.537 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:36 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9[64256]: 2024-04-25T00:20:36.170+0000 7f6f0d600700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:08.499352+0000 front 2024-04-25T00:20:08.499375+0000 (oldest deadline 2024-04-25T00:20:33.799494+0000) 2024-04-25T00:20:37.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:36 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:36.573+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:37.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:36 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:36.573+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:37.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:36 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:36.573+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:37.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:36 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:36.573+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:37.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:36 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:36.573+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:37.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:36 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:36.573+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:37.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:36 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:36.725+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:37.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:36 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:36.725+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:37.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:36 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:36.725+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:37.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:36 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:36.725+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:37.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:36 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:36.725+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:37.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:36 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:36.725+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:37.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:36 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:36.725+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:37.453 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:37 smithi110 podman[94416]: 2024-04-25 00:20:37.190450883 +0000 UTC m=+5.579065758 container died e20472883720cca2793d8409e2fd920a99b58344e947f2493dbff63844c4b74f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T00:20:37.787 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:37 smithi110 podman[94416]: 2024-04-25 00:20:37.552424799 +0000 UTC m=+5.941039682 container cleanup e20472883720cca2793d8409e2fd920a99b58344e947f2493dbff63844c4b74f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T00:20:37.787 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:37 smithi110 bash[94416]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9 2024-04-25T00:20:38.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:37 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:37.593+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:38.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:37 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:37.593+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:38.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:37 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:37.593+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:38.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:37 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:37.593+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:38.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:37 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:37.593+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:38.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:37 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:37.593+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:38.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:37 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:37.711+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:38.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:37 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:37.711+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:38.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:37 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:37.711+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:38.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:37 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:37.711+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:38.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:37 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:37.711+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:38.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:37 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:37.711+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:38.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:37 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:37.711+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:38.164 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:37 smithi110 podman[94434]: 2024-04-25 00:20:37.845477897 +0000 UTC m=+0.656960346 container remove e20472883720cca2793d8409e2fd920a99b58344e947f2493dbff63844c4b74f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-25T00:20:38.481 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:38 smithi110 podman[94526]: 2024-04-25 00:20:38.067426736 +0000 UTC m=+0.017805847 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:20:38.481 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:38 smithi110 podman[94526]: 2024-04-25 00:20:38.196314214 +0000 UTC m=+0.146693335 container create 16733bd953923301f26b693aa56263845ea5dc57890fa994b1b73a1ae5d9af42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T00:20:38.481 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:38 smithi110 podman[94526]: 2024-04-25 00:20:38.40740532 +0000 UTC m=+0.357784409 container init 16733bd953923301f26b693aa56263845ea5dc57890fa994b1b73a1ae5d9af42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T00:20:38.482 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:38 smithi110 podman[94526]: 2024-04-25 00:20:38.410823709 +0000 UTC m=+0.361202795 container start 16733bd953923301f26b693aa56263845ea5dc57890fa994b1b73a1ae5d9af42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:20:38.482 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:38 smithi110 podman[94526]: 2024-04-25 00:20:38.481618817 +0000 UTC m=+0.431997912 container attach 16733bd953923301f26b693aa56263845ea5dc57890fa994b1b73a1ae5d9af42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T00:20:38.786 INFO:journalctl@ceph.osd.9.smithi110.stdout:Apr 25 00:20:38 smithi110 podman[94526]: 2024-04-25 00:20:38.695564872 +0000 UTC m=+0.645943967 container died 16733bd953923301f26b693aa56263845ea5dc57890fa994b1b73a1ae5d9af42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T00:20:39.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:38 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:38.618+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:39.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:38 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:38.618+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:39.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:38 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:38.618+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:39.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:38 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:38.618+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:39.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:38 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:38.618+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:39.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:38 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:38.618+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:39.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:38 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:38.726+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:39.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:38 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:38.726+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:39.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:38 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:38.726+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:39.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:38 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:38.726+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:39.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:38 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:38.726+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:39.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:38 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:38.726+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:39.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:38 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:38.726+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:39.366 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T00:20:39.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:20:39.404 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T00:20:39.404 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T00:20:39.405 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.10 2024-04-25T00:20:39.717 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 systemd[1]: Stopping Ceph osd.10 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:20:40.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:39 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:39.576+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:40.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:39 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:39.576+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:40.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:39 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:39.576+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:40.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:39 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:39.576+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:40.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:39 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:39.576+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:40.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:39 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:39.576+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:40.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.717+0000 7ff654656700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.717+0000 7ff654656700 -1 osd.10 90 *** Got signal Terminated *** 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.717+0000 7ff654656700 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.734+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.734+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.734+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.734+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.734+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.734+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:40.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:39 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:39.734+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:41.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:40 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:40.588+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:41.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:40 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:40.588+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:41.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:40 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:40.588+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:41.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:40 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:40.588+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:41.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:40 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:40.588+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:41.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:40 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:40.588+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:41.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:40 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:40.766+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:41.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:40 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:40.766+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:41.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:40 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:40.766+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:41.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:40 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:40.766+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:41.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:40 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:40.766+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:41.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:40 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:40.766+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:41.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:40 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:40.766+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:42.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:41 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:41.557+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:42.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:41 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:41.557+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:42.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:41 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:41.557+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:42.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:41 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:41.557+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:42.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:41 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:41.557+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:42.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:41 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:41.557+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:42.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:41 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:41.753+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:42.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:41 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:41.753+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:42.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:41 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:41.753+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:42.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:41 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:41.753+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:42.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:41 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:41.753+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:42.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:41 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:41.753+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:42.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:41 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:41.753+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:43.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:42 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:42.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:43.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:42 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:42.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:43.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:42 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:42.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:43.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:42 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:42.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:43.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:42 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:42.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:43.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:42 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:42.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:43.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:42 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:42.519+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:43.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:42 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:42.739+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:43.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:42 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:42.739+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:43.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:42 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:42.739+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:43.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:42 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:42.739+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:43.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:42 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:42.739+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:43.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:42 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:42.739+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:43.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:42 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:42.739+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:44.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:43 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:43.548+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:44.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:43 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:43.548+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:44.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:43 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:43.548+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:44.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:43 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:43.548+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:44.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:43 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:43.548+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:44.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:43 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:43.548+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:44.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:43 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:43.548+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:44.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:43 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:43.764+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:44.102 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:43 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:43.764+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:44.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:43 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:43.764+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:44.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:43 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:43.764+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:44.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:43 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:43.764+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:44.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:43 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:43.764+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:44.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:43 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:43.764+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:45.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:44 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:44.547+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:45.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:44 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:44.547+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:45.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:44 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:44.547+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:45.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:44 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:44.547+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:45.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:44 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:44.547+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:45.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:44 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:44.547+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:45.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:44 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:44.547+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:45.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:44 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:44.717+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:20.778321+0000 front 2024-04-25T00:19:20.778289+0000 (oldest deadline 2024-04-25T00:19:46.678337+0000) 2024-04-25T00:20:45.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:44 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:44.717+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:31.180592+0000 front 2024-04-25T00:19:31.180557+0000 (oldest deadline 2024-04-25T00:19:54.080765+0000) 2024-04-25T00:20:45.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:44 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:44.717+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.482464+0000 front 2024-04-25T00:19:40.482324+0000 (oldest deadline 2024-04-25T00:20:06.382372+0000) 2024-04-25T00:20:45.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:44 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:44.717+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:48.584203+0000 front 2024-04-25T00:19:48.584219+0000 (oldest deadline 2024-04-25T00:20:13.284009+0000) 2024-04-25T00:20:45.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:44 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:44.717+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:53.284440+0000 front 2024-04-25T00:19:53.284419+0000 (oldest deadline 2024-04-25T00:20:17.984506+0000) 2024-04-25T00:20:45.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:44 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:44.717+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:04.986257+0000 front 2024-04-25T00:20:04.986164+0000 (oldest deadline 2024-04-25T00:20:25.486022+0000) 2024-04-25T00:20:45.103 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:44 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10[57344]: 2024-04-25T00:20:44.717+0000 7ff650c4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.110:6822 osd.6 since back 2024-04-25T00:20:12.487856+0000 front 2024-04-25T00:20:12.487982+0000 (oldest deadline 2024-04-25T00:20:36.587766+0000) 2024-04-25T00:20:45.484 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:45 smithi173 podman[69298]: 2024-04-25 00:20:45.175268862 +0000 UTC m=+5.576782518 container died 5a4b8ebf30231079a9c7a9ebc378a06d7f2933dd2425b4c571378300bd6a5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T00:20:45.484 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:45 smithi173 podman[69298]: 2024-04-25 00:20:45.484361981 +0000 UTC m=+5.885875694 container cleanup 5a4b8ebf30231079a9c7a9ebc378a06d7f2933dd2425b4c571378300bd6a5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:20:45.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:45 smithi173 bash[69298]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10 2024-04-25T00:20:45.852 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:45 smithi173 podman[69317]: 2024-04-25 00:20:45.68555494 +0000 UTC m=+0.507799008 container remove 5a4b8ebf30231079a9c7a9ebc378a06d7f2933dd2425b4c571378300bd6a5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:20:46.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:45 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:45.571+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:46.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:45 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:45.571+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:46.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:45 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:45.571+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:46.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:45 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:45.571+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:46.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:45 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:45.571+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:46.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:45 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:45.571+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:46.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:45 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:45.571+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:46.264 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:45 smithi173 podman[69407]: 2024-04-25 00:20:45.901889769 +0000 UTC m=+0.018421199 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:20:46.264 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:46 smithi173 podman[69407]: 2024-04-25 00:20:46.011449743 +0000 UTC m=+0.127981188 container create 60b70b26df7a99f4a1ab06634410c8315673093cefd576175ed942690bcb46ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:20:46.264 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:46 smithi173 podman[69407]: 2024-04-25 00:20:46.189114565 +0000 UTC m=+0.305645965 container init 60b70b26df7a99f4a1ab06634410c8315673093cefd576175ed942690bcb46ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T00:20:46.264 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:46 smithi173 podman[69407]: 2024-04-25 00:20:46.192173983 +0000 UTC m=+0.308707470 container start 60b70b26df7a99f4a1ab06634410c8315673093cefd576175ed942690bcb46ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T00:20:46.264 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:46 smithi173 podman[69407]: 2024-04-25 00:20:46.264000339 +0000 UTC m=+0.380531746 container attach 60b70b26df7a99f4a1ab06634410c8315673093cefd576175ed942690bcb46ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:20:46.602 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 25 00:20:46 smithi173 podman[69407]: 2024-04-25 00:20:46.479141459 +0000 UTC m=+0.595672863 container died 60b70b26df7a99f4a1ab06634410c8315673093cefd576175ed942690bcb46ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T00:20:47.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:46 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:46.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:47.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:46 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:46.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:47.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:46 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:46.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:47.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:46 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:46.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:47.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:46 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:46.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:47.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:46 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:46.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:47.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:46 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:46.590+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:47.085 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T00:20:47.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:20:47.123 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T00:20:47.123 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T00:20:47.123 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.11 2024-04-25T00:20:47.420 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 systemd[1]: Stopping Ceph osd.11 for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:20:47.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.419+0000 7f51e7406700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:47.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.419+0000 7f51e7406700 -1 osd.11 90 *** Got signal Terminated *** 2024-04-25T00:20:47.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.419+0000 7f51e7406700 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:20:47.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.551+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:47.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.551+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:47.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.551+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:47.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.551+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:47.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.551+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:47.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.551+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:47.769 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:47 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:47.551+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:49.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:48.518+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:49.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:48.518+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:49.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:48.518+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:49.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:48.518+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:49.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:48.518+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:49.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:48.518+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:49.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:48 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:48.518+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:49.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:49.517+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:49.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:49.517+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:49.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:49.517+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:49.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:49.517+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:49.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:49.517+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:49.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:49.517+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:49.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:49 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:49.517+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:51.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:50.561+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:51.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:50.561+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:51.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:50.561+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:51.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:50.561+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:51.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:50.561+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:51.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:50.561+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:51.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:50 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:50.561+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:52.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:51.592+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T00:19:21.648779+0000 front 2024-04-25T00:19:21.648680+0000 (oldest deadline 2024-04-25T00:19:46.948716+0000) 2024-04-25T00:20:52.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:51.592+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-25T00:19:32.249958+0000 front 2024-04-25T00:19:32.250036+0000 (oldest deadline 2024-04-25T00:19:53.949813+0000) 2024-04-25T00:20:52.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:51.592+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-25T00:19:40.351581+0000 front 2024-04-25T00:19:40.351389+0000 (oldest deadline 2024-04-25T00:20:05.651535+0000) 2024-04-25T00:20:52.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:51.592+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.110:6814 osd.3 since back 2024-04-25T00:19:47.952716+0000 front 2024-04-25T00:19:47.952776+0000 (oldest deadline 2024-04-25T00:20:11.253488+0000) 2024-04-25T00:20:52.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:51.592+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6812 osd.4 since back 2024-04-25T00:19:51.253975+0000 front 2024-04-25T00:19:51.253952+0000 (oldest deadline 2024-04-25T00:20:21.854432+0000) 2024-04-25T00:20:52.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:51.592+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-25T00:20:01.854959+0000 front 2024-04-25T00:20:01.855057+0000 (oldest deadline 2024-04-25T00:20:26.555055+0000) 2024-04-25T00:20:52.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:51.592+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.173:6820 osd.7 since back 2024-04-25T00:20:17.557452+0000 front 2024-04-25T00:20:17.557574+0000 (oldest deadline 2024-04-25T00:20:42.257449+0000) 2024-04-25T00:20:52.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:51 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11[58348]: 2024-04-25T00:20:51.592+0000 7f51e39fa700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-04-25T00:20:25.158303+0000 front 2024-04-25T00:20:25.158331+0000 (oldest deadline 2024-04-25T00:20:51.058314+0000) 2024-04-25T00:20:53.115 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:52 smithi188 podman[70554]: 2024-04-25 00:20:52.8297949 +0000 UTC m=+5.509572951 container died 4532eb270b10819d3cfd5c0f9a139b86bb919409ef3fc46a6a5540f5c2a1300e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T00:20:53.462 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:53 smithi188 podman[70554]: 2024-04-25 00:20:53.203109577 +0000 UTC m=+5.882887624 container cleanup 4532eb270b10819d3cfd5c0f9a139b86bb919409ef3fc46a6a5540f5c2a1300e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T00:20:53.462 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:53 smithi188 bash[70554]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11 2024-04-25T00:20:53.718 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:53 smithi188 podman[70572]: 2024-04-25 00:20:53.462584096 +0000 UTC m=+0.629279593 container remove 4532eb270b10819d3cfd5c0f9a139b86bb919409ef3fc46a6a5540f5c2a1300e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T00:20:54.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:53 smithi188 podman[70663]: 2024-04-25 00:20:53.666861513 +0000 UTC m=+0.017640564 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:20:54.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:53 smithi188 podman[70663]: 2024-04-25 00:20:53.771977404 +0000 UTC m=+0.122756480 container create 786f3aa1d4084edac4a18e2015718ed8ceb238b29f7b8935a6be6a3f588be2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T00:20:54.288 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:54 smithi188 podman[70663]: 2024-04-25 00:20:54.024257239 +0000 UTC m=+0.375036273 container init 786f3aa1d4084edac4a18e2015718ed8ceb238b29f7b8935a6be6a3f588be2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:20:54.288 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:54 smithi188 podman[70663]: 2024-04-25 00:20:54.027529181 +0000 UTC m=+0.378308223 container start 786f3aa1d4084edac4a18e2015718ed8ceb238b29f7b8935a6be6a3f588be2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T00:20:54.288 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:54 smithi188 podman[70663]: 2024-04-25 00:20:54.157739951 +0000 UTC m=+0.508518991 container attach 786f3aa1d4084edac4a18e2015718ed8ceb238b29f7b8935a6be6a3f588be2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T00:20:54.610 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 25 00:20:54 smithi188 podman[70663]: 2024-04-25 00:20:54.33516509 +0000 UTC m=+0.685944127 container died 786f3aa1d4084edac4a18e2015718ed8ceb238b29f7b8935a6be6a3f588be2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T00:20:54.879 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T00:20:54.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:20:54.920 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T00:20:54.920 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T00:20:54.920 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.a 2024-04-25T00:20:55.238 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 25 00:20:54 smithi110 systemd[1]: Stopping Ceph mds.a for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:20:55.500 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 25 00:20:55 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a[70694]: 2024-04-25T00:20:55.237+0000 7fa020d42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:20:55.500 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 25 00:20:55 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a[70694]: 2024-04-25T00:20:55.237+0000 7fa020d42700 -1 mds.a *** got signal Terminated *** 2024-04-25T00:20:59.221 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 25 00:20:58 smithi110 podman[94645]: 2024-04-25 00:20:58.969183157 +0000 UTC m=+3.852650552 container died bf534df8e285c50c84ceca4568650ac2129d457bdd77de38cca88e4aa900571a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T00:20:59.496 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 25 00:20:59 smithi110 podman[94645]: 2024-04-25 00:20:59.303684527 +0000 UTC m=+4.187151926 container cleanup bf534df8e285c50c84ceca4568650ac2129d457bdd77de38cca88e4aa900571a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T00:20:59.497 INFO:journalctl@ceph.mds.a.smithi110.stdout:Apr 25 00:20:59 smithi110 bash[94645]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-a 2024-04-25T00:20:59.707 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T00:20:59.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:20:59.743 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T00:20:59.744 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T00:20:59.744 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.d 2024-04-25T00:21:00.057 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 25 00:20:59 smithi110 systemd[1]: Stopping Ceph mds.d for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:00.536 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 25 00:21:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d[69690]: 2024-04-25T00:21:00.056+0000 7fc23f45d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:00.536 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 25 00:21:00 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d[69690]: 2024-04-25T00:21:00.056+0000 7fc23f45d700 -1 mds.d *** got signal Terminated *** 2024-04-25T00:21:03.184 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 25 00:21:02 smithi110 podman[94776]: 2024-04-25 00:21:02.916702725 +0000 UTC m=+2.955293840 container died 9759b01d5621230d7c945f9a307bedc43d4188071d844cf3da80106afb184766 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T00:21:03.536 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 25 00:21:03 smithi110 podman[94776]: 2024-04-25 00:21:03.282426272 +0000 UTC m=+3.321017376 container cleanup 9759b01d5621230d7c945f9a307bedc43d4188071d844cf3da80106afb184766 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:21:03.537 INFO:journalctl@ceph.mds.d.smithi110.stdout:Apr 25 00:21:03 smithi110 bash[94776]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-d 2024-04-25T00:21:03.702 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T00:21:03.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:03.734 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T00:21:03.735 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T00:21:03.735 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.g 2024-04-25T00:21:04.037 INFO:journalctl@ceph.mds.g.smithi110.stdout:Apr 25 00:21:03 smithi110 systemd[1]: Stopping Ceph mds.g for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:04.537 INFO:journalctl@ceph.mds.g.smithi110.stdout:Apr 25 00:21:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-g[70303]: 2024-04-25T00:21:04.061+0000 7f3827b6e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:04.537 INFO:journalctl@ceph.mds.g.smithi110.stdout:Apr 25 00:21:04 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-g[70303]: 2024-04-25T00:21:04.062+0000 7f3827b6e700 -1 mds.g *** got signal Terminated *** 2024-04-25T00:21:08.195 INFO:journalctl@ceph.mds.g.smithi110.stdout:Apr 25 00:21:07 smithi110 podman[94896]: 2024-04-25 00:21:07.907537436 +0000 UTC m=+3.960997554 container died 3178ef81328bd39f503d2e4df20cd4c996c583ef0df9b0cc1c28ae59c1adf2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-g, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T00:21:08.528 INFO:journalctl@ceph.mds.g.smithi110.stdout:Apr 25 00:21:08 smithi110 podman[94896]: 2024-04-25 00:21:08.285475776 +0000 UTC m=+4.338935893 container cleanup 3178ef81328bd39f503d2e4df20cd4c996c583ef0df9b0cc1c28ae59c1adf2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T00:21:08.528 INFO:journalctl@ceph.mds.g.smithi110.stdout:Apr 25 00:21:08 smithi110 bash[94896]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-g 2024-04-25T00:21:08.648 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T00:21:08.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:08.680 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T00:21:08.680 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T00:21:08.680 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.j 2024-04-25T00:21:08.786 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 25 00:21:08 smithi110 systemd[1]: Stopping Ceph mds.j for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:09.286 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 25 00:21:08 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j[69340]: 2024-04-25T00:21:08.980+0000 7f01996b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:09.286 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 25 00:21:08 smithi110 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j[69340]: 2024-04-25T00:21:08.980+0000 7f01996b3700 -1 mds.j *** got signal Terminated *** 2024-04-25T00:21:12.192 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 25 00:21:11 smithi110 podman[95018]: 2024-04-25 00:21:11.901548643 +0000 UTC m=+3.006069611 container died 1712dc0d25d90930e6b9176bddd77c9bec20ffe05b8a27e5f5a0ec13bb4b0380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:21:12.457 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 25 00:21:12 smithi110 podman[95018]: 2024-04-25 00:21:12.272509853 +0000 UTC m=+3.377030820 container cleanup 1712dc0d25d90930e6b9176bddd77c9bec20ffe05b8a27e5f5a0ec13bb4b0380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T00:21:12.457 INFO:journalctl@ceph.mds.j.smithi110.stdout:Apr 25 00:21:12 smithi110 bash[95018]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-j 2024-04-25T00:21:12.681 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T00:21:12.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:12.712 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T00:21:12.712 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T00:21:12.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.b 2024-04-25T00:21:13.029 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:21:12 smithi173 systemd[1]: Stopping Ceph mds.b for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:13.352 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:21:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: 2024-04-25T00:21:13.028+0000 7f3412d63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:13.352 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:21:13 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b[62867]: 2024-04-25T00:21:13.028+0000 7f3412d63700 -1 mds.b *** got signal Terminated *** 2024-04-25T00:21:15.054 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:21:14 smithi173 podman[69525]: 2024-04-25 00:21:14.79949089 +0000 UTC m=+1.892057630 container died 29446a726e8df71e64420a032bb72629d9dc7d56fd875b4b5568dad25abba700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T00:21:15.352 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:21:15 smithi173 podman[69525]: 2024-04-25 00:21:15.153171099 +0000 UTC m=+2.245737847 container cleanup 29446a726e8df71e64420a032bb72629d9dc7d56fd875b4b5568dad25abba700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:21:15.352 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 25 00:21:15 smithi173 bash[69525]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-b 2024-04-25T00:21:15.581 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T00:21:15.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:15.615 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T00:21:15.615 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T00:21:15.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.e 2024-04-25T00:21:15.852 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:21:15 smithi173 systemd[1]: Stopping Ceph mds.e for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:16.352 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:21:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e[62253]: 2024-04-25T00:21:15.948+0000 7fa69bed8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:16.352 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:21:15 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e[62253]: 2024-04-25T00:21:15.948+0000 7fa69bed8700 -1 mds.e *** got signal Terminated *** 2024-04-25T00:21:19.091 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:21:18 smithi173 podman[69645]: 2024-04-25 00:21:18.792057906 +0000 UTC m=+2.962188494 container died a4b4b4bffc1b70f2a72be2c5db17d27833f82d67ebbe19c46490bf356f02e55d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:21:19.352 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:21:19 smithi173 podman[69645]: 2024-04-25 00:21:19.198773177 +0000 UTC m=+3.368903776 container cleanup a4b4b4bffc1b70f2a72be2c5db17d27833f82d67ebbe19c46490bf356f02e55d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-25T00:21:19.352 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 25 00:21:19 smithi173 bash[69645]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-e 2024-04-25T00:21:19.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T00:21:19.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:19.588 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T00:21:19.588 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T00:21:19.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.h 2024-04-25T00:21:20.102 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:21:19 smithi173 systemd[1]: Stopping Ceph mds.h for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:20.103 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:21:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h[62559]: 2024-04-25T00:21:19.910+0000 7f51193f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:20.103 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:21:19 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h[62559]: 2024-04-25T00:21:19.910+0000 7f51193f8700 -1 mds.h *** got signal Terminated *** 2024-04-25T00:21:23.086 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:21:22 smithi173 podman[69778]: 2024-04-25 00:21:22.801571137 +0000 UTC m=+2.991649290 container died 1172453ed4452b36b17a03010626cb35194d13e81417dabaa36ed7ff4bc51329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T00:21:23.352 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:21:23 smithi173 podman[69778]: 2024-04-25 00:21:23.168781834 +0000 UTC m=+3.358859987 container cleanup 1172453ed4452b36b17a03010626cb35194d13e81417dabaa36ed7ff4bc51329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T00:21:23.353 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 25 00:21:23 smithi173 bash[69778]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-h 2024-04-25T00:21:23.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T00:21:23.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:23.574 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T00:21:23.574 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T00:21:23.574 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.k 2024-04-25T00:21:23.852 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:23 smithi173 systemd[1]: Stopping Ceph mds.k for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:24.352 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:23 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k[61936]: 2024-04-25T00:21:23.897+0000 7f2b3a763700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:24.352 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:23 smithi173 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k[61936]: 2024-04-25T00:21:23.897+0000 7f2b3a763700 -1 mds.k *** got signal Terminated *** 2024-04-25T00:21:25.223 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:24 smithi173 podman[69899]: 2024-04-25 00:21:24.962547299 +0000 UTC m=+1.175045902 container died 54140fa474372d057f069f26fccf674dcd6071781731d09d7e6be6eb1814d680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T00:21:25.490 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:25 smithi173 podman[69899]: 2024-04-25 00:21:25.297655407 +0000 UTC m=+1.510154026 container cleanup 54140fa474372d057f069f26fccf674dcd6071781731d09d7e6be6eb1814d680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T00:21:25.490 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:25 smithi173 bash[69899]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k 2024-04-25T00:21:25.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T00:21:25.741 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:25 smithi173 podman[69913]: 2024-04-25 00:21:25.490332019 +0000 UTC m=+0.517321410 container remove 54140fa474372d057f069f26fccf674dcd6071781731d09d7e6be6eb1814d680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:21:25.741 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:25 smithi173 systemd[1]: ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-25T00:21:25.741 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:25 smithi173 systemd[1]: Stopped Ceph mds.k for 787fa75a-0295-11ef-bc93-c7b262605968. 2024-04-25T00:21:25.741 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 25 00:21:25 smithi173 systemd[1]: ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.061s CPU time. 2024-04-25T00:21:25.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:25.751 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T00:21:25.751 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T00:21:25.752 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.c 2024-04-25T00:21:26.067 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:21:25 smithi188 systemd[1]: Stopping Ceph mds.c for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:26.518 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:21:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c[63497]: 2024-04-25T00:21:26.066+0000 7f17f4343700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:26.518 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:21:26 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c[63497]: 2024-04-25T00:21:26.066+0000 7f17f4343700 -1 mds.c *** got signal Terminated *** 2024-04-25T00:21:30.417 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:21:30 smithi188 podman[70769]: 2024-04-25 00:21:30.079730254 +0000 UTC m=+4.132402260 container died 05ee9d634df9d94bda33935d5b8868554eb512110af4d6a6a419fc9b783ab920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T00:21:30.684 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:21:30 smithi188 podman[70769]: 2024-04-25 00:21:30.491272957 +0000 UTC m=+4.543944962 container cleanup 05ee9d634df9d94bda33935d5b8868554eb512110af4d6a6a419fc9b783ab920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T00:21:30.684 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 25 00:21:30 smithi188 bash[70769]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-c 2024-04-25T00:21:30.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T00:21:30.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:30.851 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T00:21:30.851 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T00:21:30.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.f 2024-04-25T00:21:31.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:21:30 smithi188 systemd[1]: Stopping Ceph mds.f for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:31.518 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:21:31 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f[63189]: 2024-04-25T00:21:31.253+0000 7fa1aa04f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:31.518 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:21:31 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f[63189]: 2024-04-25T00:21:31.253+0000 7fa1aa04f700 -1 mds.f *** got signal Terminated *** 2024-04-25T00:21:35.161 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:21:34 smithi188 podman[70902]: 2024-04-25 00:21:34.830399756 +0000 UTC m=+3.764260950 container died 3ceaca1cfe29790e3189b1bb75d8da64a38939364c617dcdb8e39f10eddbbb14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T00:21:35.161 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:21:35 smithi188 podman[70902]: 2024-04-25 00:21:35.160964251 +0000 UTC m=+4.094825452 container cleanup 3ceaca1cfe29790e3189b1bb75d8da64a38939364c617dcdb8e39f10eddbbb14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T00:21:35.512 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:21:35 smithi188 bash[70902]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f 2024-04-25T00:21:35.512 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 25 00:21:35 smithi188 podman[70916]: 2024-04-25 00:21:35.403965088 +0000 UTC m=+0.568690412 container remove 3ceaca1cfe29790e3189b1bb75d8da64a38939364c617dcdb8e39f10eddbbb14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:21:35.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T00:21:35.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:35.568 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T00:21:35.568 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T00:21:35.568 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.i 2024-04-25T00:21:35.890 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:21:35 smithi188 systemd[1]: Stopping Ceph mds.i for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:36.270 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:21:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i[63805]: 2024-04-25T00:21:35.889+0000 7f7283864700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:36.270 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:21:35 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i[63805]: 2024-04-25T00:21:35.889+0000 7f7283864700 -1 mds.i *** got signal Terminated *** 2024-04-25T00:21:39.115 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:21:38 smithi188 podman[71021]: 2024-04-25 00:21:38.835101646 +0000 UTC m=+3.054221075 container died e29352a6f3e51dc692d648818e2fc5081f11cacddad5c70c15c8d16244e04144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T00:21:39.440 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:21:39 smithi188 podman[71021]: 2024-04-25 00:21:39.20641415 +0000 UTC m=+3.425533569 container cleanup e29352a6f3e51dc692d648818e2fc5081f11cacddad5c70c15c8d16244e04144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:21:39.441 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 25 00:21:39 smithi188 bash[71021]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-i 2024-04-25T00:21:39.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T00:21:39.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:39.618 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T00:21:39.618 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T00:21:39.618 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.l 2024-04-25T00:21:40.018 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 25 00:21:39 smithi188 systemd[1]: Stopping Ceph mds.l for 787fa75a-0295-11ef-bc93-c7b262605968... 2024-04-25T00:21:40.018 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 25 00:21:39 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l[62878]: 2024-04-25T00:21:39.925+0000 7fc3f9c5c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:21:40.018 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 25 00:21:39 smithi188 ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l[62878]: 2024-04-25T00:21:39.925+0000 7fc3f9c5c700 -1 mds.l *** got signal Terminated *** 2024-04-25T00:21:42.342 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 25 00:21:42 smithi188 podman[71141]: 2024-04-25 00:21:42.028038652 +0000 UTC m=+2.196082985 container died 5a226839493fa2c419b252abdee8b638abf90c4b3604a95deeeef6f3ce132f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:21:42.342 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 25 00:21:42 smithi188 podman[71141]: 2024-04-25 00:21:42.342797972 +0000 UTC m=+2.510842311 container cleanup 5a226839493fa2c419b252abdee8b638abf90c4b3604a95deeeef6f3ce132f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-25T00:21:42.768 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 25 00:21:42 smithi188 bash[71141]: ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l 2024-04-25T00:21:42.768 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 25 00:21:42 smithi188 podman[71155]: 2024-04-25 00:21:42.577303497 +0000 UTC m=+0.546526858 container remove 5a226839493fa2c419b252abdee8b638abf90c4b3604a95deeeef6f3ce132f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-787fa75a-0295-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T00:21:42.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-787fa75a-0295-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T00:21:42.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:21:42.855 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T00:21:42.855 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 787fa75a-0295-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:21:43.042 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-25T00:21:45.583 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 787fa75a-0295-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:21:45.769 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-25T00:21:48.201 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 787fa75a-0295-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:21:48.389 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-25T00:21:50.875 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:50.907 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:50.939 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:21:50.965 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T00:21:50.967 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi110/crash 2024-04-25T00:21:50.968 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:21:51.009 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:21:51.010 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:21:51.011 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi173/crash 2024-04-25T00:21:51.012 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:21:51.055 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:21:51.055 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:21:51.057 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi188/crash 2024-04-25T00:21:51.058 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:21:51.102 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/787fa75a-0295-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:21:51.102 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:21:51.104 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T00:21:51.104 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T00:21:51.142 INFO:tasks.cephadm:Compressing logs... 2024-04-25T00:21:51.142 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:21:51.185 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:21:51.187 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:21:51.221 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T00:21:51.221 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-25T00:21:51.222 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:21:51.222 INFO:teuthology.orchestra.run.smithi110.stderr:gzip/var/log/ceph/ceph-client.0.log: -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr: 86.9% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:21:51.228 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.a.log: 82.9% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:21:51.229 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T00:21:51.229 INFO:teuthology.orchestra.run.smithi110.stderr: 90.2% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:21:51.229 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T00:21:51.229 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:21:51.230 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T00:21:51.230 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:21:51.230 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:21:51.230 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T00:21:51.231 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T00:21:51.233 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T00:21:51.233 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T00:21:51.234 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:21:51.235 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T00:21:51.235 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:21:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T00:21:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:gzip/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T00:21:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:21:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:21:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T00:21:51.240 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:21:51.290 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 83.0% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:21:51.317 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T00:21:51.330 INFO:teuthology.orchestra.run.smithi173.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:21:51.347 INFO:teuthology.orchestra.run.smithi110.stderr: 94.7% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T00:21:51.369 INFO:teuthology.orchestra.run.smithi110.stderr: 94.7% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T00:21:51.377 INFO:teuthology.orchestra.run.smithi173.stderr: 90.4% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:21:51.379 INFO:teuthology.orchestra.run.smithi173.stderr: 86.9% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:21:51.387 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T00:21:51.429 INFO:teuthology.orchestra.run.smithi173.stderr: 93.2% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T00:21:51.468 INFO:teuthology.orchestra.run.smithi173.stderr: 91.0% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:21:51.475 INFO:teuthology.orchestra.run.smithi188.stderr: 91.1% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:21:51.497 INFO:teuthology.orchestra.run.smithi110.stderr: 91.1% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:21:51.537 INFO:teuthology.orchestra.run.smithi110.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:21:51.581 INFO:teuthology.orchestra.run.smithi110.stderr: 94.7% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T00:21:51.638 INFO:teuthology.orchestra.run.smithi173.stderr: 94.6% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T00:21:51.643 INFO:teuthology.orchestra.run.smithi188.stderr: 82.9% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:21:51.719 INFO:teuthology.orchestra.run.smithi188.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:21:51.730 INFO:teuthology.orchestra.run.smithi188.stderr: 90.4% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:21:51.743 INFO:teuthology.orchestra.run.smithi188.stderr: 86.9% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:21:51.752 INFO:teuthology.orchestra.run.smithi188.stderr: 94.7% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T00:21:52.022 INFO:teuthology.orchestra.run.smithi188.stderr: 93.5% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T00:21:52.244 INFO:teuthology.orchestra.run.smithi110.stderr: 90.2% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T00:21:53.002 INFO:teuthology.orchestra.run.smithi173.stderr: 91.9% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T00:21:53.472 INFO:teuthology.orchestra.run.smithi173.stderr: 96.2% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T00:21:53.514 INFO:teuthology.orchestra.run.smithi110.stderr: 90.5% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T00:21:55.009 INFO:teuthology.orchestra.run.smithi188.stderr: 91.9% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T00:22:00.362 INFO:teuthology.orchestra.run.smithi173.stderr: 95.5% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T00:22:07.185 INFO:teuthology.orchestra.run.smithi188.stderr: 94.5% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T00:22:16.101 INFO:teuthology.orchestra.run.smithi110.stderr: 91.2% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T00:22:51.030 INFO:teuthology.orchestra.run.smithi110.stderr: 90.7% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi173.stderr: 91.1% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T00:23:08.312 INFO:teuthology.orchestra.run.smithi173.stderr: 97.2% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T00:23:39.644 INFO:teuthology.orchestra.run.smithi173.stderr: 90.7% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T00:23:45.535 INFO:teuthology.orchestra.run.smithi173.stderr: 92.0% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T00:23:53.281 INFO:teuthology.orchestra.run.smithi173.stderr: 91.1% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T00:23:53.283 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-04-25T00:23:53.283 INFO:teuthology.orchestra.run.smithi173.stderr:real 2m2.082s 2024-04-25T00:23:53.283 INFO:teuthology.orchestra.run.smithi173.stderr:user 3m57.029s 2024-04-25T00:23:53.283 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m9.223s 2024-04-25T00:26:37.955 INFO:teuthology.orchestra.run.smithi110.stderr: 90.8% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T00:27:28.283 INFO:teuthology.orchestra.run.smithi110.stderr: 90.6% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T00:29:33.645 INFO:teuthology.orchestra.run.smithi188.stderr: 91.1% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T00:29:40.502 INFO:teuthology.orchestra.run.smithi110.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-25T00:29:40.503 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-04-25T00:29:40.504 INFO:teuthology.orchestra.run.smithi110.stderr:real 7m49.304s 2024-04-25T00:29:40.504 INFO:teuthology.orchestra.run.smithi110.stderr:user 5m49.815s 2024-04-25T00:29:40.504 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m14.534s 2024-04-25T00:30:54.334 INFO:teuthology.orchestra.run.smithi188.stderr: 91.0% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T00:30:56.416 INFO:teuthology.orchestra.run.smithi188.stderr: 91.0% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T00:31:26.476 INFO:teuthology.orchestra.run.smithi188.stderr: 91.1% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T00:35:33.345 INFO:teuthology.orchestra.run.smithi188.stderr: 94.5% -- replaced with /var/log/ceph/787fa75a-0295-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T00:35:33.346 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-04-25T00:35:33.346 INFO:teuthology.orchestra.run.smithi188.stderr:real 13m42.145s 2024-04-25T00:35:33.347 INFO:teuthology.orchestra.run.smithi188.stderr:user 7m16.031s 2024-04-25T00:35:33.347 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m18.154s 2024-04-25T00:35:33.348 INFO:tasks.cephadm:Archiving logs... 2024-04-25T00:35:33.349 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi110/log 2024-04-25T00:35:33.350 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:36:33.887 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi173/log 2024-04-25T00:36:33.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:37:11.411 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi188/log 2024-04-25T00:37:11.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:38:20.143 INFO:tasks.cephadm:Removing cluster... 2024-04-25T00:38:20.143 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 787fa75a-0295-11ef-bc93-c7b262605968 --force 2024-04-25T00:38:20.430 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-25T00:38:20.952 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 787fa75a-0295-11ef-bc93-c7b262605968 --force 2024-04-25T00:38:21.243 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-25T00:38:21.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 787fa75a-0295-11ef-bc93-c7b262605968 --force 2024-04-25T00:38:22.020 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 787fa75a-0295-11ef-bc93-c7b262605968 2024-04-25T00:38:22.645 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T00:38:22.645 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:38:22.663 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:38:22.681 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:38:22.696 INFO:tasks.cephadm:Teardown complete 2024-04-25T00:38:22.696 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:38:22.707 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:38:22.707 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:38:22.710 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:38:22.724 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:38:22.885 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:38:22.896 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:38:22.896 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:38:22.900 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:38:22.902 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:38:22.914 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:38:22.916 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:38:22.916 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:38:22.920 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:38:22.920 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-04-25T00:38:22.920 INFO:teuthology.orchestra.run.smithi110.stdout:^- hv01.front.sepia.ceph.com 2 6 377 44 +964us[ +964us] +/- 572ms 2024-04-25T00:38:22.920 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -49us[ -33us] +/- 53ms 2024-04-25T00:38:22.920 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv03.front.sepia.ceph.com 3 6 377 46 +63us[ +79us] +/- 61ms 2024-04-25T00:38:22.920 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:38:22.924 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:38:22.924 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-25T00:38:22.924 INFO:teuthology.orchestra.run.smithi173.stdout:^- hv01.front.sepia.ceph.com 2 6 377 41 +970us[ +970us] +/- 572ms 2024-04-25T00:38:22.925 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 -50us[ -50us] +/- 53ms 2024-04-25T00:38:22.925 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv03.front.sepia.ceph.com 3 6 377 48 +55us[ +61us] +/- 61ms 2024-04-25T00:38:22.925 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:38:22.942 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:38:22.942 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-04-25T00:38:22.942 INFO:teuthology.orchestra.run.smithi188.stdout:^- hv01.front.sepia.ceph.com 2 6 377 51 +959us[ +947us] +/- 572ms 2024-04-25T00:38:22.942 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 -65us[ -65us] +/- 53ms 2024-04-25T00:38:22.942 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv03.front.sepia.ceph.com 3 6 377 48 +47us[ +34us] +/- 61ms 2024-04-25T00:38:22.942 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:38:22.943 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:38:22.953 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:38:22.954 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:38:22.965 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:38:23.088 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:38:23.197 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:38:23.289 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:38:23.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:23.319 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:38:23.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:23.349 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:38:23.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:23.377 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:38:23.388 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:38:23.406 INFO:teuthology.task.internal:Duration was 3175.099378 seconds 2024-04-25T00:38:23.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:38:23.426 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:38:23.426 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:38:23.429 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:38:23.431 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:38:23.481 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:38:23.488 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:38:23.507 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:38:23.900 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:38:23.901 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T00:38:23.901 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:38:23.925 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-04-25T00:38:23.925 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:38:23.957 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T00:38:23.957 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:38:23.980 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:38:23.980 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:38:23.983 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:38:24.000 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:38:24.045 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:38:24.046 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:38:24.053 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:38:24.066 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:38:24.433 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:38:24.446 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:38:24.446 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:38:24.478 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:38:24.521 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:38:24.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:38:24.562 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:38:24.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:38:24.567 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:38:24.585 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-04-25T00:38:24.595 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-04-25T00:38:24.614 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-04-25T00:38:24.656 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:38:24.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:24.671 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:38:24.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:24.690 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:38:24.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:38:24.710 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:38:24.722 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:38:24.723 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi110 2024-04-25T00:38:24.723 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:38:24.767 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi173 2024-04-25T00:38:24.767 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:38:24.802 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672223/remote/smithi188 2024-04-25T00:38:24.802 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:38:24.838 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:38:24.838 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:38:24.841 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:38:24.843 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:38:24.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:38:24.902 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:38:24.902 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:38:24.913 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:38:24.914 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:38:24.916 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:38:24.918 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:38:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:38 /home/ubuntu/cephtest 2024-04-25T00:38:24.930 INFO:teuthology.orchestra.run.smithi173.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:38 /home/ubuntu/cephtest 2024-04-25T00:38:24.942 INFO:teuthology.orchestra.run.smithi188.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:38 /home/ubuntu/cephtest 2024-04-25T00:38:24.943 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:38:24.957 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:38:24.984 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 3175.099378347397 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:38:24.984 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:38:25.061 INFO:teuthology.run:pass